Category: General

XMPP-FTW now supports Superfeedr

By , Sunday 12th May 2013 5:07 pm

Summary

As of version 0.9.0 xmpp-ftw now supports the Superfeedr XMPP API. If you don’t know what Superfeedr is then read this shamelessly stolen description from Crunchbase:

Superfeedr fetches and parses RSS or Atom feeds on behalf of its users and then pushes them the new entries in these feeds. Superfeedr implements most of the current Real-time technologies and guarantees an entry detection time inferior to 15 min. Superfeedr has both an XMPP and a PubSubHubbub API.Read more: http://www.crunchbase.com/company/superfeedr#ixzz2T6A0Grml

The XMPP-FTW interface to Superfeedr is built off their documentation which can be found here: http://superfeedr.com/documentation#pubsubhubbub.

Continue reading 'XMPP-FTW now supports Superfeedr'»

XMPP For the Web (XMPP-FTW)

By , Wednesday 20th March 2013 9:12 pm

I’d like to introduce you to my latest project XMPP-FTW. The name is a (hopefully) clever play on “For The Win” (FTW) but actually I call it “XMPP For The Web”.

Essentially XMPP-FTW tries to make XMPP in the browser as quick and painless as many of the other solutions for realtime web by translating XML to JSON and back and using named events to help fill in the missing pieces.

The project is open source and the code is available on github at XMPP-FTW source code, you can also view the manual or play with a demo on XMPP-FTW website.

Continue reading 'XMPP For the Web (XMPP-FTW)'»

New demo system for XMPP-FTW

By , Sunday 10th March 2013 6:39 pm

Originally seen on http://awesome-wildlife.blogspot.co.uk/2009/12/aardvark.html

I’ve spent most of the day writing a new demo system for XMPP-FTW and despite it looking ugly as sin (I am no god with design) I’m quite pleased with how it works, so I thought I’d write up a little piece about it…

Continue reading 'New demo system for XMPP-FTW'»

An error has occurred: {“bytesParsed”:0,”code”:”HPE_INVALID_CONSTANT”}

By , Saturday 12th January 2013 2:38 pm

From bdc.co.uk

I’ve been writing a new application in Nodejs, using websockets (socket.io), this application is deployed using the PaaS Nodejitsu.  Everything has been going great and I’ve been surprised how easy it has been to create a realtime application using socket.io. Deployment has also been a breeze with Nodejitsu’s tools.  I develop on a Linux machine myself but the other day I passed the details to someone using a windows machine running internet explorer.  The application stopped working with an error message, a redeploy didn’t help. The error I was presented with was as follows:;

An error has occurred: {“bytesParsed”:0,”code”:”HPE_INVALID_CONSTANT”}

As I haven’t uploaded any new code in about a week I made the incorrect assumption that something had gone wrong on nodejitsu’s side and so dropped them a tweet to let them know as they are still in beta as far as I understand.

Within 90 minutes I’d got a reply from Nuno Job (@dscape) from Nodejitsu letting me know that there was an issue with Internet Explorer, socket.io, and Joyent’s servers. He also included a workaround, and a link with further details. Excellent support!

From what I’ve understood the proxies used at Joyent don’t like non-HTTP response (from the flashsocket) and so prevent any further connections to the domain (please correct me in the comments if incorrect).

The solution is to turn off flashsocket as a transport when configuring socket.io as follows:

var io = require('socket.io').listen(80);

io.configure(function(){
    io.set('transports', [
        'websocket',
        'htmlfile',
        'xhr-polling',
        'jsonp-polling'
    ]);
});

See: https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO

I hope this helps anyone that comes across the same issue, the original solution/explanation came from http://blog.dreamflashstudio.com/2012/08/nodejitsu-on-joyent/.

With this in place the app sprung back into life. I just really need to report to users with browsers that don’t implement websockets that they need to use something more modern…

 

 

 

 

Why you should refuse to eat meat slaughtered according to Halal/Kosher methods

comments Comments Off
By , Thursday 9th February 2012 7:41 pm

This isn’t the usual sort of thing I write about on my site, but it is something I feel passionately about. I’ve recently come across this video that, quite frankly, sickened me to the depths of my stomach and feel that it should be shared with as many people as possible. This is in order to educate the public each time they make a decision about consuming animal produced via the practice of ritual slaughter…

 

Despite my girlfriend being a vegan I am still a meat eater, I understand that in order to put meat on my plate animals have to die. I’m not here to argue whether it is moral/ethical to eat meat, I do that’s not going to change. I believe strongly however that if an animal has to die in order to feed people then it should live a healthy, comfortable, relaxed life and be treated with enough decency that when the time comes for it to be slaughtered it should be done in a way which causes the least amount of discomfort and suffering.

 

In the UK (as well as most other countries) we have laws which govern the welfare of animals during the slaughtering process. Sadly these laws have exemptions for religious slaughtering methods which, I believe, subject the animals to undue stress and suffering (I’ll leave you to judge this for yourself). This isn’t an attack against religion or religious people (believe in what you want, just don’t cause hurt/harm to anyone else) but using religion to justify this barbaric treatment of animals is unacceptable to me. I would rather suffer in purgatory than pass into the heavenly kingdom of any god which allows this level of pain and suffering to another creature (except I know that none these things exist). Those sanctifying these practices as part of their religion are just as accountable for what horrific acts are performed in the name of their religion as those performing them.

 

Due to the reduction in costs an abattoir incurs by circumventing the general animal welfare laws by using ritual slaughter methods, halal/kosher meat (and animal by-products) are commonly used throughout a number of brands. For example, Nestle, Kellog’s (I’ve recently had to give up my favourite breakfast cereal), Lyle’s golden syrup, Kingsmill, just to name a few. Additionally what I would consider a mark of quality products, the ‘Assured Food Standards’ (Red Tractor logo), seems to suggest that, actually, what you are eating is ritually slaughtered meat. Having looked into this further the number of products containing halal/kosher products is astounding.

 

Ultimately I would like to see an end to this practice but I realise that this probably isn’t economically viable for companies who also produce products for Islamic countries or to appeal to the UK Islamic population. What I would like to see is a marking on food products which contain meat (or animal by-products) which have been slaughtered using inhumane methods i.e. halal or kosher slaughter. I would also extend this to restaurants or any place serving food.  I believe this marking would allow myself and others to make an informed choice when it comes to the products I choose to purchase, and the practices I choose to support.

 

One such scheme has a petition here to sign:  http://www.causes.com/causes/538272-boycott-halal/actions/1725485

 

Thank you for reading (if you’ve got this far), I’ll now finish with a copy of the video I’ve talked about, be aware that this is quite disturbing.  The video starts by showing the generally accepted ‘bolt method’ slaughtering process used in most abattoirs in the UK. It then goes on to show the halal/kosher slaughtering method and the suffering of the animals before, during, and – most disturbingly – after the slaughtering has taken place.

 

Please watch it and understand what has happened to an animal (or animals) when you choose to eat, or refuse to acknowledge/check that the food you are consuming, has been slaughtered according to Halal/Kosher methods. I believe the original source of this video is French, but I’m sure the process is similar in the UK. I hope the video has the same affect on you that it did to me and changes your perception of such animal slaughtering methods in the future as well as your purchasing habits…

 

 

PHP Design Patterns – Observer Pattern

By , Tuesday 29th December 2009 10:02 pm

I’ve been reading Head First Design Patterns recently and have decided to write some of the patterns as PHP examples for my own benefit. The first one that I’ve decided to code up is the Observer Pattern. The formal definition of the Observer Pattern is:

The observer pattern (a subset of the asynchronous publish/subscribe pattern) is a software design pattern in which an object, called the subject, maintains a list of its dependents, called observers, and notifies them automatically of any state changes, usually by calling one of their methods. It is mainly used to implement distributed event handling systems.

As systems become more loosely coupled making sure that when an event happens all systems that require knowledge of these updates are informed. For example, a blog post, after saving a post we may need to update a search engine (e.g. Lucene), update our sitemap, tags, email subscribed users, etc. The observer pattern allows developers to add additional listeners without editing their observable object. By injecting observers (i.e. a search engine update observer, a sitemap generator, etc) into a subject (i.e. blog post editing system) we can allow the it to perform all the necessary updates without any changes.

Continue reading 'PHP Design Patterns – Observer Pattern'»

Office Grid Computing using Virtual environments – Part 3

By , Friday 4th December 2009 11:37 pm

Introduction

I work in a company where we run many batch jobs processing millions of records of data each day and I’ve been thinking recently about all the machines that sit around each and every day doing nothing for several hours. Wouldn’t it be good if we could use those machines to bolster the processing power of our systems? In this set of articles I’m going to look at the potential benefits of employing an office grid using virtualised environments.

In part 2 we looked at the jobs a server will run, and how jobs should be configured in order to achieve greatest amount of processing whilst ensuring that each job is processed without fail.

Setting up your worker – or LiMP server

The next step in the process is to set up your virtual workers. For this I’m going to use an installation of centOS using VirtualBox. I’m going to install mySQL and PHP on the server, also known as a LiMP (Linux, mySQL, PHP) Server  (I may have made that name up).

  • Install VirtualBox on your windows machine (follow link)
  • Download and install centOS (current version 5.3) within a created virtual machine

There’s no point me going to this there’s probably 1,000′s of great tutorials out there (ok, here’s one: Creating and Managing  centOS virtual machine under virtualbox). The important point to note I suppose is that I called my virtual machine GridMachine.

As far as my choices of virtualisation client and operating system go there is no big compelling reason for each choice. VirtualBox is something I use on my home machine and is supported by the three major operating systems. I chose centOS as its a good stable OS and I use it on my own web server. I am a great believer in the right tools for the job (although I’m applying ‘use the quickest and easiest for you’ mentality here), so if operating system X runs your code quicker and more efficiently use that instead :)

Importantly make sure that your VM uses DHCP, otherwise for each new virtual machine would need to be configured separately which is something we don’t want.By using DHCP we don’t need to configure network settings individually for worker machines, DHCP will hand out IPs for you. Therefore you can copy your virtual machine about the office without worrying about setting each one up (this improves scalability and reduces worker administration).

The process you should aim to achieve would be to obtain a new physical machine, install VirtualBox, and then pretty much deploy the virtual image without much else. It might be wise to setup all your workers on a different subnet so that you can at least see how many machines are running. You’ll also need to set up your machines on a long lease or unlimited lease DHCP.

How to run Jobs on the worker

This is an interesting area and there are several valid methods for processing jobs on the worker. Here I’ll just discuss the two most obvious:

  • Perpetually running script: A script, be it a shell script, or a PHP script is executed once on the worker and runs as part of an infinite loop. I’ve discounted this method as one crash of the script and potentially your workers will cease to run without some sort of intervention.
  • Cron based script execution: Every X minutes the cron daemon kicks off a call to your script to get things going. Without some checking this could lead to many many copies of your worker script running.

My decision was to go with cron which kicks off a shell script every 10 minutes.  My shell script performs the following tasks:

  1. Get a process list and grep this for ‘php’. If not found then continue.
  2. Call your job code, in my case this would be something PHP based
  3. Worker script completes its run
  4. Ready to go again on the next appropriate call

My bash script looks something like the following:

#!/bin/sh
if ps ax | grep -v grep | grep php > /dev/null
then
    echo "Job is currently processing, exit"
else
    echo "Job is not running, start now"
    php yourJobProcessingScript.php
fi

Note: the echo’s are almost completely pointless, but may help the next person who comes along to try and edit them.

That concludes the set up of the worker virtual machine, quick, simple, and easy to copy to each new piece of hardware that is received. The ‘cleverness’ of the grid system really isn’t in the visualised OS, its all to do with the code created to process jobs, the job configuration, and in making sure that the job runs when appropriate (i.e. when the host is idle).

Setting up Windows to Initialise Workers

The first task is to work out the command required to run the virtual machine from the windows command line. If you’ve installed virtualBox in the default location and you’ve named your worker GridMachine then the command required to load up your worker is:

"C:\Program Files\Sun\VirtualBox\VBoxManage.exe" startvm GridMachine

However to run the script in a ‘headless’ state we need to use:

"C:\Program Files\Sun\VirtualBox\VBoxHeadless.exe" -startvm GridMachine --vrdp=off

This will start the virtual machine without the GUI and allow it to save state gracefully. The second argument turns off RDP so it doesn’t conflict with windows RDP, or give you a message about listening on port 3389. The virtual machine name is cAsE sEnSiTiVe!

Next, we’ll need to set windows up to kick off our worker VM once the machine has been idle. To do this (on Windows XP) you’ll need to go Start -> All Programs -> Accessories -> System Tools -> Scheduled Tasks as below:

scheduled tasks

Next click on ‘Add Scheduled Task’ followed by browse to add a custom program. Navigate to your VBoxManage script and click ok. Schedule your task for any of the options (we’ll change this in a minute) and continue. After skipping the next screen windows will ask you who you want to run this task, I’d suggest either ‘Administrator’ or creating a new privileged user. Remember we don’t want to interfere with the standard staff account on the machine at any point. Click next and check show advanced options for this task.

To the end of the run textbox add our ‘startvm GridMachine‘ string and ensure that run only when logged in is left unticked. Visit the schedule task next and change the schedule drop down to the option ‘when idle’, choose the amount of time you’d like the machine to be idle before moving on to the next tab.

Finally untick the option which states stop the task if it has been running X amount of time, but do tick the option to stop the task if the machine is no longer idle.

schedule

That’s it then for the windows host setup!

Summary

In this part we have set up a virtual machine to act as a worker, as well as the way in which we call and execute our job processing scripts (for myself a PHP script). From here we look at how to set up our copies of windows to start up the virtual machine in headless mode when the computer becomes idle, and save its state when the user resumes usage of the machine. Hopefully at this point you’re seeing how simple it is to set up such a system and are itching to get some experiments going yourself!

Next time

In Part 4 we’ll be looking at using tools to ensure that you’re running the latest version of the code and data sources so that obtained results are always up-to-date with the latest business information and logic.

UK Names Directory and Facebook Application

By , Friday 10th April 2009 9:33 am

One of the fun things I’ve been involved in whilst working for Tracesmart is the names directory. Basically we took our huge information database and pulled out a big pile of stats about various names.

The process is all one big SEO effort in order to draw more traffic to the site, but it also provides some interesting information and name statistics.

Here’s an example for my name Lloyd Watkin, or I suppose more correctly Steven Watkin. Take a minute to look up your name on the Tracesmart site, might be interesting ;)

We’ve also got names statistics data on the names directory page which tells you what names have been searched and how recently. Its quite interesting to see famous people’s names fly up the stats when they appear in the news. A recent example is that of Liam Neeson who’s wife died from a skiing accident recently, although obviously people aren’t quite right on the spelling (hence the huge numbers of hits on this name) — Liam Nilson.

Facebook Application

Once the names directory was created we set about creating a facebook application which we duely named ‘My Name‘. The facebook application is an extension of the names directory and provides some additional statistics. In order to add the names directory to your facebook account please visit:

http://apps.facebook.com/my-name/

Flickr and Yahoo!: Forgotten Login Details

By , Thursday 2nd April 2009 7:31 pm

A few months ago I changed my Flickr password so that a friend could upload some shots to it. A couple of months later I find I’ve forgotten that password and trying to remember all my secret details from Yahoo! is just a nightmare. I had my Yahoo! email address for several years and I wasn’t sure when or where (I was living) when I signed up – oh just to make this clearer Flickr uses Yahoo! login system :)

Yahoo! were no help either, they wanted exact details and would reject anything that didn’t match. Plus I wasn’t allowed to provide a list of possible matching details it had to be right on every detail otherwise it would be rejected. I can understand the security but this was getting a pain in the arse, especially as I could retrieve my user name to my usual email address without any trouble (why couldn’t you send a reset there!?!?!). On top of all this they wanted the details sent to their offices in California either by mail or fax, boo!

Step back to Flickr who kindly send an email to my registered email address asking me to detach my Yahoo! account from my Flickr account. Hooray I could then sign up with a new Yahoo! account and reattach my Flickr account to this new login.

After waiting at least a week each time for Yahoo! to reply Flickr got this all sorted within 2-3 hours!

Well done Flickr, thankfully you’re out there looking after your customers :)

Now for everyone’s boredom here’s yet another link to my Flickr account

Inspired… and guilted!

By , Wednesday 11th March 2009 8:51 pm

I’ve recently set up a blog for one of the guys in work – Matthew Hopkins – and seeing his first few posts has made me think that maybe I should add a few posts as its been a good while and lots has happened over 2008. Another one of the guys in work does himself a blog to, good old Stephen Griffiths so take a look there too.

So I’m going to plod on and add some retrospective posts about what 2008 held for me and how 2009 is shaping up :)

Panorama Theme by Themocracy

3 visitors online now
0 guests, 3 bots, 0 members
Max visitors today: 8 at 07:37 am UTC
This month: 12 at 15-09-2014 02:03 pm UTC
This year: 79 at 09-06-2014 07:22 pm UTC
All time: 130 at 28-03-2011 10:40 pm UTC