Evilprofessor.co.uk

Zend Framework Body Tag View Helper

Steven Lloyd Watkin — Sat, 21 Aug 2010 23:13:28 +0000

Photo from http://www.flickr.com/photos/daniello/

Summary

Here I discuss the creation of a view helper for modifying HTML tags, and more specifically body tags. The created view helper allows functionality similar to the head*/inlineScript view helpers already in the standard Zend Framework view helpers, but allows the programmatic modification of tag attributes. Definitely check out the demo page and the code on github.

Motivation

The standard Zend Framework view helpers are a great set of tools for streamlining mundane view tasks and allowing for the modification/addition of scripts and header blocks (generally held in the layout) from within the view without applying ugly hacks (i.e. the head*/inlineScript view helpers).

Upon occasion I have found need to make modifications to the tag, for example adding an onload, class, or style attribute etc. I also required to be able to perform this from within other view helpers. Take this following contrived example…

On website X, certain pages include standard dojo forms. These dojo forms are held within view helpers for convenience. Generally it has been decided not to include the dojo CSS classes in the body tag and only add them when necessary. There maybe several view helpers on the page that need to add their own attributes to the body tag. (I said it was contrived)

The code is available in my GIT repository @ github and the demo page.

How it works

When creating the body tag view helper I soon realised that most of the code was not specific to what I was developing so I pulled out the common functionality into an abstract class. By concreting the abstract class it is possible to modify any tag in the same way as the body tag discussed here making the code much more versitile.

Underneath the hood (so to speak) the code is based on that of the head* and the inlineScript view helpers. It utilises Zend_View_Helper_Placeholder_Container_Standalone to store the attributes and their values until they are required. The container itself uses Zend_Registry to store values under the key defined in the class (if you look at the unit tests I clear the registry down after each test).

The code would fit in well with the “Naked Zend_Layout and Zend_View” code I wrote about a couple of weeks ago or as part of the standard Zend Framework MVC install.

Usage

The class should be very simple to use. Firstly two entries are required within your layout script as follows:

bodyTag() ?>
bodyTag()->toString(Pro_View_Helper_AbstractTag::CLOSE) ?>

Note: You’ll need to change the class name should you move the view helper to your own library. Obviously the constant is available through the implementation (bodyTag) as well as through the abstract class.

An example of making changes to your body tag using the view helper to echo is:

bodyTag('class', 'nihilo')
    ->bodyTag('style', 'text-align: left')
    ->bodyTag('style', 'width: 80em', false, ';')
    ->bodyTag('class', 'myclass')
?>

This can be done throughout the view file, in several view helpers, and in the layout file (provided its before the body tag is written out). Attributes can be removed/overwritten whilst values can be added and appended (separator can be passed). If the same attribute value is passed several times only a single copy is written with the attribute.

For a full demo either fork the github repository and run the demo.php file in your own browser, or alternatively visit this page to see it in action. Usage of the demo.php script assumes you have the Zend Autoloader (or similar) already setup.

Creating your own Tag helper

To implement your own view helper simply extend the abstract tag class and overwrite the following class properties:

$_regKey: A unique key for the tag (used for Zend_Registry)
$_validAttributes: An array of valid attribute names, should be lowercase to comply with W3C
$_selfClosing: Boolean value to specify if tag is self closing or not, e.g.
$_tagName: Name of the tag (i.e. body in this example)

So, finally the code for the bodyTag view helper itself, its very compact:

/**
 * View helper for the body tag
 *
 * @author     Lloyd Watkin
 * @since      21/08/2010
 * @package    Pro
 * @subpackage ViewHelper
 */
class Pro_View_Helper_BodyTag
    extends Pro_View_Helper_AbstractTag
{
    /**
     * Registry key for placeholder
     *
     * @var string
     */
    protected $_regKey = 'Pro_View_Helper_BodyTag';

    /**
     * Which attributes are valid
     *
     * Currently only STF attributes supported
     * (S = STRICT, T = TRANSITIONAL, F = FRAMESET)
     *
     * @see http://www.w3schools.com/tags/tag_body.asp
     * @var array
     */
    protected $_validAttributes = array(
        /* Standard Attributes */
        'class', 'dir', 'id', 'lang', 'style', 'title', 'xml:lang',
        /* Event Attributes */
        'onclick', 'ondblclick', 'onload', 'onmousedown', 'onmousemove',
        'onmouseout', 'onmouseover', 'onmouseup', 'onkeydown', 'onkeypress',
        'onkeyup', 'onunload',
    );

    /**
     * Self closing tag?
     *
     * @var boolean
     */
    protected $_selfClosing = false;    

    /**
     * Tag name
     *
     * @var string
     */
    protected $_tagName = 'body';
}

If you take a look at the repository there’s also a group of unit tests that fully cover the functionality. The unit tests are written in PHPUnit and are in the BodyTagTest.php file.

Finally…

Here I’ve shown you my implementation of an abstract tag abstract view helper (eek) and its body tag implementation. Whilst the javaScript functionality can be implemented in other ways (other than hard coding at output time) other things can’t (without javaScript that is) without changing other parts of the application. I hope you find the classes helpful in your application…

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Quick Start Symfony DI (Dependency Injection) Tutorial

Steven Lloyd Watkin — Sat, 14 Aug 2010 14:21:35 +0000

What is Dependency Injection (DI)?

Dependency injection is a technique that allows for loosely coupled objects within a software application. Generally if an object requires access to the functionality of another it would be instantiated internally leading to tightly coupled systems. By implementing dependency injection we inject the required objects ready for use (sometimes also referred to inversion of control – IOC). Take the following example:

getScore($parameters);
        /* ... Some more decision logic ... */
        return ($parameterScore > 50);
    }
}

This piece of code is said to be tightly coupled to the DecisionParameters object. Rewriting the above in a loosely coupled fashion we’d have something like….

_dp = $dp;
    }
    public function makeDecision(array $parameters) {
        $parameterScore = $this->_dp->getScore($parameters);
        /* ... Some more decision logic ... */
        return ($parameterScore > 50);
    }
}

Whilst gaining the benefits of loosely coupled code we are adding complexity such that each time an object is instantiated we also have to instantiate its dependencies and pass these in too. For example, this:

$choice = new DecisionMaker();
echo $choice->makeDecision(array('effort' => 'low', 'return' => 'high'));

now becomes:

$dp = new DecisionParameters();
$choice = new DecisionMaker($dp);
echo $choice->makeDecision(array('effort' => 'low', 'return' => 'high'));

This situation becomes more painful as the number of dependencies for a class is increased, and what if the dependencies themselves have dependencies? This can quite quickly become an object administration nightmare! Enter dependency injection containers (or frameworks)…

Dependency Injection Containers/Frameworks

Dependency injection containers (or frameworks) handle the process of object creation; instantiating and injecting any dependencies before returning an instance to the caller.

In your code rather than create new objects directly we request a copy of the object from the DI container. The object we are returned already has all its dependencies injected and the object is ready to go.

Symfony Dependency Injection Container

Symfony is probably best known for their full stack MVC framework however they have also released a number of the components that can be used independently. For example, the dependency injection container we’re going to talk about here, a YAML parser, a templating engine, see Symfony components for more.

The Symfony DI container is based on that from the Spring Framework in Java.

Bootstrapping

In order to bootstrap the Symfony dependency injection framework we use code as included below. I have chosen to use YAML just because its easy to read and setup. For maximum speed you may want to write out your containers to plain PHP (the Symfony container can do this for you once setup), or alternatively cache the whole container using Zend_Cache, or similar.

To install Symfony DI follow the instructions included here http://components.symfony-project.org/dependency-injection/installation, and add it to your path.

// Load the Symfony DI container
require 'sfServiceContainerBuilder.php';
$container = new sfServiceContainerBuilder();
$loader = new sfServiceContainerLoaderFileYaml($container);
$loader->load(APPLICATION_PATH . '/config/di/services.yml');

Firstly we instantiate a new container, and then we load our configuration from a YAML file. Note: the DI container can load config from several formats such as XML, YAML, PHP, and INI*. I tend to include a single YAML file and import other files from within there.

Several configuration files can be imported using different formats, the newer definitions overwriting those that have already been defined. Configuration files can include references to objects and parameters.

*INI is only able to define parameters and is unable to import other files

Configuration Example

imports:
  - { resource: daos.yml }

parameters:
  username: false

services:
  # Customer model
  model.customer:
    class:  Pro_Customer
    calls:
      - [ setLogger, [ @utils.logger ] ]
      - [ setDao, [ @data.userdata.mysql ] ]
      - [ setUserName, [ %username% ] ]

  # Product model
  model.product:
    class:  Pro_Product
    arguments: [%username%, { type: %accesslevel%, lastlogin: %lastlogin% } ]
    calls:
     - [ setDao, [ @data.product.mysql ] ]

  # Logger
  utils.logger:
    class: Pro_Logger
    constructor: getInstance
    calls:
      - [ setHandle, [ @utils.filewriter ] ]

I think the code above is fairly self explanatory but for clarity I’ll explain each part now.

First off we define some imports (i.e. other files to parse), I like to group my configurations for example DAOs in one file, utilities in another and name the file appropriately. Whilst a little slower it speeds up the maintenance of the configuration files. Its also possible to parse files of other formats using different import flags. Files are parsed in order with newer definitions overwriting or amending previously defined services/parameters.

Next we define a parameter, a parameter can generally be any PHP variable type. At this point I didn’t know what my username parameter should be (I need to authenticate for that!), so I’ve defined a default value and I’ll overwrite that value later. Note, classes aren’t instantiated until you ask for them so defining parameters a little later is perfectly fine. Following this I define some services:

Instantiate Pro_Customer, pass an instance of my logger to the setLogger() method, add in my MySQL data access object (DAO), and pass in the username as well. Each time I ask for this object I want a new instance
Create an instance of Pro_Product, pass arguments of username and an options array to the constructor. After instantiation call setDao() and pass it my product DAO
Give me a copy of Pro_Logger, instantiate it using the getInstance() method and pass a copy of my file writer object via setHandle() once its loaded. My file writer is defined in one of my imports.

Within the configuration files services are referenced by prepending the name with an ‘@’ symbol, parameters are referenced by prepending and appending with ‘%’ symbols, e.g. @utils.logger %username%.

Adding data post loading

Sometimes you don’t know what the value of parameters should be until after you have bootstrapped, as with our username parameter above. In order to overwrite the value the offsetSet() method is used, firstly passing the parameter name followed by its new value:

$container->offsetSet('username', $username);

Getting Objects from the container

Once setup rather than instantiating objects directly we now go to the DI container to get instances. With the Symfony DI we call the getService() method, passing in a string which describes the variable you wish to retrieve, e.g.

$logger = $container->getService('utils.logger');

This will perform the equivalent of:

$logger = Pro_Logger::getInstance();
$fileWriter = new Pro_Writer_FileWriter();
$logger->setHandler($fileWriter);

It is possible to determine if the container has an instance of a service by calling the hasService() method, which returns a boolean value.

Its not necessary to have access to the container beyond the top level of your application since once retrieving an instance all of the required dependencies right down to the deepest depths of your application are already setup and waiting to be called.

Unit Testing

Dependency injection also has the added benefit of allowing objects to be tested in isolation. Taking the DecisionMaker class example before implementing the dependency injection setup our test results were also dependent on the implementation of the DecisionParameters class.

Should the DecisionParameters class return different results our tests could start failing through no fault of our own. Using dependency injection we can now pass in a DecisionParameters object which returns a known/fixed set of results for certain input parameters, we are now testing DecisionMaker in isolation from any external factors, i.e. if our tests start failing this can be directly attributed to something changing in the DecisionMakeer class. This is particularly important should you be relying on test data from a database for testing.

Application Configuration

The way in which your application behaves on your production server will inevitably differ from your development setup. For example, in a development environment the level of logging would be much more granular than on your production system. By altering your dependency injection container configuration files slightly (or parsing an additional configuration file) the altering of behaviour depending on the environment is made simple.

By type hinting on interfaces rather than implementations when setting your dependencies its also possible to swap out components for compatible components with a few lines of YAML or XML. For instance, you may currently access your database via a MySQL DAO (Data Access Object), but in time you may develop a PDO, Zend_Db, or Doctrine implementation of this DAO. By adding the new implementation into your configuration file suddenly all the objects that used the old MySQL implementation are now using your new implementation without needing to attack the bowels of your application.

Notes

By setting shared: true in the configuration we are always given the same instance of an object. This is very useful when dealing with objects which contain resources such as database connections, file handles, etc
There are two methods by which dependencies can be injected. One is to pass dependencies with the constructor, or alternatively via setter methods. The generally accepted pattern is to pass required dependencies via the constructor and optionals via setters. My personal preference is to use setters for everything, but this is down to the individual developer

Finally….

Along with a brief discussion on dependency injection and its advantages and disadvantages this has also been a quick-start guide to implementing the Symfony Dependency Injection Container… hopefully you’ve seen that it’s surprisingly quick and easy. From here you’ll be able to start using dependency injection through your application and look to using many more of the advanced features (although the simple example covers the vast majority of functionality you’ll require). As always I point you to the manual for further information.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Naked Zend_Layout and Zend_View

Steven Lloyd Watkin — Tue, 10 Aug 2010 23:47:28 +0000

In this article I look at using Zend_Layout and Zend_View along with a simple front controller to show how it is possible to start separating business logic and presentation within your application. All code is available on github:
Naked Zend_Layout and Zend_View on GitHub.

MVC

A common design pattern for modern web applications is the MVC pattern. The Zend Framework in ‘full stack’ mode is one implementation of MVC in PHP and consists of three parts:

Model (M)
View (V)
Controller (C)

The pattern is designed such that business and presentation logic are completely separated from one another (with business logic in the model, and presentation in the view) and the controller sitting in the middle conducting the “orchestra”.

Presentation and Logic

In the Zend Framework the view is handled by two main components: Zend_View and Zend_Layout. Zend_Layout as the name suggests looks after the layout aspect of the site (generally headers, footers, sidebars, etc). Zend_View focusses on presenting the data that your model has been working to produce or derive.

As developers, and their applications, evolve we tend to move through various stages, generally each an improvement on the previous, improving maintainability and extensibility. One of the main issues is that presentation and logic still get intermingled and its not simple to start separating the two.

What’s wrong with mixing the two?

There’s several reasons why mixing different parts of the application, for example a designer working on your site may not want to (or have the knowledge) to scan around in code trying to work out where to make presentation changes. In the same way a developer (if you’re like me who has the design skills of an ostrich) may break out in a cold sweat when you mention design or UI work.

Additionally what if later you would like to present your sites on different media, such as mobile telephones, tablet PCs, or expose the data via web services (XML/JSON/etc)? Having mixed presentation and logic you stand almost no hope without some very ugly hacks to pull the presentation back out of your code, before injecting something new. If data and presentation has been separated making these changes are almost trivial, create a new view script for the new format and direct requests as appropriate.

Separating the two

In an evolving application its not always economical to start implementing a full MVC solution and the application needs to be migrated slowly – sometimes running old code in parallel with new. It maybe that there are masses of logic (such as database connection setup, authentication, cookie handling, etc) that aren’t ready for moulding to your chosen frameworks setup, therefore old and known-to-be-working code can continue to be used until such time it can be rewritten/refactored.

Note: Using Zend_Layout and Zend_View like this is perfectly acceptable within the Zend Framework landscape and the framework has been designed such that individual components can be used without the rest of the framework. A great advantage in evolving applications and probably one of the major reasons for its high uptake in enterprise applications.

Front Controller

Below I create a front controller - a single file designed to pick up any requests which are not matched to a file on the file-system. This is often achieved using a .htaccess file such as the one used in the default Zend Framework install. Within the front controller I will be setting up our layout and view and showing where the different parts of the application slip into it.

define('APP_PATH', dirname(__FILE__). '/..');
// Start buffering output
ob_start();

// Create a Zend_View instance
Zend_Layout::startMvc();
$layout = Zend_Layout::getMvcInstance();
$layout->setLayoutPath(APP_PATH . '/layout/scripts')
    ->setViewSuffix('phtml')
    ->setLayout('index');

$view = $layout->getView()
    ->setScriptPath(APP_PATH . '/view/scripts')
    ->addHelperPath(APP_PATH . '/library/Zend/View/Helper', 'Zend_View_Helper');

// Set Base URL - ok *almost* naked, but you don't need this!
Zend_Controller_Front::getInstance()->setBaseUrl($_SERVER['HTTP_HOST']);

try {
    /**
     * Perform some application routing...
     *  - Could be using this as a front controller and directing all requests
     *    through this one file (provided file does not exist in file system
     *  - Note the method below is only really for demonstration, it would be
     *    horrible with a large site
     */
    switch ($_GET['page']) {
    	case 'index':
    	case 'exception':
    		$pageName = $_GET['page'];
    		break;
    	default:
    		$pageName = false;
    		break;
    }
    // Example of a page not being found...
    if (false === $pageName) {
        $responseHeader = 'HTTP/1.1 404 Page Not Found';
        throw new Exception('Page not Found');
    }

    /**
     * Add data to your view object here
     *   You may have your own controller implementation or some includes files
     *   where business logic is partly separated from view logic
     */
    $view->displayText = 'Hello from Lloyd';
    $view->buttonText  = 'I\'m not active!';

	$layout->content = $view->render("{$pageName}.phtml");
    echo $layout->render();
} catch (Exception $e) {
	// Clean out already buffered content - we don't want to display that!
	ob_clean();
    if (!isset($responseHeader)) {
    	$responseHeader = 'HTTP/1.1 500 Internal Server Error';
    }
    header($responseHeader);
    $view->exception = $e;
    $layout->content = $view->render('error.phtml');
    echo $layout->render();
}

Firstly we start output buffering, by doing this we can set our headers at any point in the request and know that it is possible to send them. Should an exception be thrown at any stage of the code execution we clean this buffer and write out or error message content and layout. This ensures that we do not deliver part rendered content containing errors to the end user.

Next a new MVC instance of Zend_Layout is generated and we tell it that out layout scripts have the extension phtml, are to be found in a directory outside of the public path, and that our default layout is called index(.phtml). From the layout we then extract the view object (to which we set our data to be presented) and apply similar setup.

Next we setup the view object with a reference to the default Zend_View helpers. View helpers are sets of additional convenience functionality. For example, writing out a float in monetary format, or creating a zebra striped table (they can be read about here). By extending Zend_View_Helper_Abstract and adding your own library at this point its possible to use your own application view helpers.

The rest of the application code is now wrapped in a try {} catch {} block. Should anything throw an uncaught exception we can catch it and display a nice error message to the end user.

Our first task within the try {} catch {} is to route our request, what does the user want to see? Here I’ve implemented some very simple demonstration code where I check the value of the ‘page’ get variable. Your routing can be very much more complex. The routing is used to call what ever code needs to be executed to get/handle data provided by the user and to tell the system what view (and possibly layout script) to use.

Ultimately if our router does not match any page it sets a 404 response code and displays a nice page not found message to the end user. Here, we throw and catch our own exception (and a very generic exception at that) but probably you’d be throwing your own exception from within the router code.

Once we’ve successfully routed our request we can begin to do something with the code. It might be that you have your own controllers/models implemented or you include some code that’s already been separated somewhat. Here I set a couple of simple variables to the view object.

Once this is completed we simply render the views with our data. If the code throws an uncaught exception for any reason this is trapped near the bottom of the script. Here we clear already buffered output, set a 500 response header, and tell our application to render the ‘error’ view script (which is generally a much stripped down version of the normal layout/view and logs the error for checking later).

As the view is rendered first and injected into the layout it is possible to change the layout from within the view, and indeed set required extras, for example,

Page title
Meta tags
Scripts (URLs or code) into the <head> section
Add additional styles, etc

Additionally its even possible to change the entire layout from within the view by using…

layout()->setLayout('alternativeLayout') ?>

…as and when required.

Finally…

I hope this has been a useful introduction to Zend_Layout and Zend_View and it will enable you to start implementing your own basic MVC and improve the maintainibility/extensibility of your code. Please take a look at the source code for examples of usage (see the README file for instructions).

The code assumes that you’ve already got autoloading working (or you have included the required classes). In addition, I would not recommend that you implement the routing or data setting as above, this is very much simplified for demonstration. To see the full code take a look at the source code linked at the top of this article.

Zend Framework version: 1.10.6

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Zend Framework Per Module Layout Settings – Follow Up

Steven Lloyd Watkin — Tue, 16 Feb 2010 20:48:44 +0000

As a follow up to my previous post on per module based layout settings for Zend Framework, I’ve updated the code to require less configuration then before (not that it required more that a few lines in your application configuration!).

Again we make use of a Zend Controller Action Helper invoking it from the bootstrap as follows:

    /**
     * Sets up layout scripts on a per-module basis
     */
    protected function _initLayoutHelper()
	{
	    $this->bootstrap('frontController');
	    $layout = Zend_Controller_Action_HelperBroker::addHelper(
	        new Pro_Controller_Action_Helper_SetLayoutPath(APPLICATION_PATH));
	}

Almost exactly the same as before except this time we pass the constant APPLICATION_PATH into the constructor as our base path.

Next the controller action helper itself now looks as follows:

/**
 * Sets the layout path on a per-module basis
 *
 * @author     Lloyd Watkin
 * @since      16/02/2010
 * @package    Pro
 * @subpackage Controller_Action_Helper
 */

/**
 * Sets the layout path on a per-module basis
 *
 * @author     Lloyd Watkin
 * @since      16/02/2010
 * @package    Pro
 * @subpackage Controller_Action_Helper
 */
class Pro_Controller_Action_Helper_SetLayoutPath
    extends Zend_Controller_Action_Helper_Abstract
{
	/**
	 * Base path
	 *
	 * @var string
	 */
	protected $_path;

	/**
	 * Construct
	 *
	 * @param string $path
	 */
	public function __construct($path)
	{
	    $this->setBasePath($path);
	}

	/**
	 * Set base path
	 *
	 * @param string $path
     */
	public function setBasePath($path)
	{
		if (!is_string($path) || empty($path)) {
			throw new Exception('Excepted string for base path');
		}
		$this->_path = $path;
	}

	/**
	 * Get the base path
	 *
	 * @return string
	 */
	protected function _getBasePath()
	{
		if (is_null($this->_path)) {
			if (!defined('APPLICATION_PATH')) {
				throw new Exception('Base path can not be determined');
			}
			$this->_path = APPLICATION_PATH;
		}
		return $this->_path;
	}

    /**
     * Sets layout path based on module
     */
    public function preDispatch()
    {
    	$module = preg_replace(
    	    '/[^A-Z]/i', '', $this->getRequest()->getModuleName()
    	);

	    if ($bootstrap = $this->getActionController()
	                       ->getInvokeArg('bootstrap')) {

	        $view = $bootstrap->getResource('view');
	        $layoutPath = $this->_getBasePath() .
	            "/modules/{$module}/layouts/scripts/";

	        /* If layout directory exists then apply it, otherwise just fall
	         * back on the default
	         */
	        if (is_dir($layoutPath)) {
	            $this->getActionController()
	                 ->getHelper('layout')
	                 ->setLayoutPath($layoutPath);
	            $view->headLink()->appendStylesheet(
	                "/styles/{$module}/style.css"
	            );
	        }
    	}
    }
}

Note the passing of our base path (cf. APPLICATION_PATH) in the constructor, we also have a getter and setter for the base path. Unlike the previous version of the code if the layouts path does not exist then the code will fall back the default layout path. If the base path is not set it will try to determine the base path from the constant APPLICATION_PATH, otherwise an exception is thrown. I’ve also cheekily thrown in a module-based style sheet for good measure

The module based layout is determined from whether the layout script path exists (obviously this method can be changed easily). This does add a little overhead from the application.ini setup (however if the layout path exists PHP will cache the result for performance*) but in return you gain the added convenience of more easily dropping in new modules.

That’s it, not much different, but you don’t need to add a new configuration entry each time you wish to add a new module :)

*To clear this cache (if required) use the clearstatcache function.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Creating URL in Zend Customer View Helper

Steven Lloyd Watkin — Thu, 28 Jan 2010 23:01:57 +0000

This may seem simple, but I was banging my head trying to create a URL in a custom view helper in Zend Framework. I have routing setup which gets the module from the sub-domain in use so I couldn’t use a simple hardcoded URL.

Basically but invoking an instance of the front controller its possible to grab the router and assemble a url. Assemble is the function used in the view helper. The URL is built up from an array of module, controller, action, etc, followed by a second parameter of the route to use. The code is as follows:

getRouter();

        return $router->assemble(array(
            'module'     => $module,
            'controller' => $controller,
            'action'     => $action,
            'category'   => "{$category->id}-{$category->name}",
        ), 'www-index');
    }
}

Another way to do this is to invoke Zend_View_Helper_Url itself and call the Url method (if you want to use the helper itself). This can be done by using the following code:

url(array(
            'module'     => $module,
            'controller' => $controller,
            'action'     => $action,
            'category'   => "{$category->id}-{$slug}",
        ), 'www-index');
    }
}

Both almost identical. Not a hard thing to do in the framework but can catch you out ;)

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Dynamically add pages to Zend_Navigation container at runtime

Steven Lloyd Watkin — Thu, 07 Jan 2010 22:50:19 +0000

In a continuation on my last post about Zend_Navigation, Route requests for sitemap.xml to custom controller/action, this post is about dymnamically adding pages to a Zend_Navigation container at runtime/script execution.

Its all well and good specifying your pages in a ini or xml file but at some point you’re going to have changing pages in your site that you want as part of a menu, sitemap, or to be included in your breadcrumb trail. Therefore what we need to do is add pages to our Zend_Navigation container at runtime. Examples for this would be in adding news items, blog posts, or page comments, etc.

In this example I’m going to add some news posts to my statically defined ini config. To get my news post page configurations I’ve used a class which returns an array in the following format:

$pagesToAdd = array (
  0 =>
    array (
      'label' => 'Fake news story #5...',
      'module' => 'www',
      'route' => 'www-index',
      'action' => 'view',
      'controller' => 'news',
      'params' => array (
          'id' => '5-Fake-news-story--5' )
    ),
  1 =>
    array ( /* More page details */ ),
 );

As you’ll notice that the function has returned an array in which are contained arrays which make up the config arrays for Zend_Navigation_Page_Mvc. Therefore, by looping over the array new Zend_Navigation pages can be created from the config array. The next thing to do as part of the loop is to to add the pages in the correct position (alternatively pages can be added in bulk by using ->addPages() method).

To do this, locate the page you wish to add the sub-pages to and simply add the pages. In this case I have used the following code to find my page:

$container->findOneBy('label', 'Latest News')->addPage($page);

My overall navigation initialisation in the bootstrap therefore looks like this:

    /**
     * used for handling top-level navigation
     *
     * @return Zend_Navigation
     */
    protected function _initNavigation()
    {
        $this->bootstrap('layout');
        $layout = $this->getResource('layout');
        $view = $layout->getView();
        $config = new Zend_Config_Ini(
            APPLICATION_PATH . '/configs/navigation.ini', APPLICATION_ENV);

        $container = new Zend_Navigation($config->default);
        // Now add the last 25 news reports
        $news  = new News();
        $pages = $news->getNavigationEntries();
        foreach ($pages AS $page) {
        	$page = new Zend_Navigation_Page_Mvc($page);
        	$container->findOneBy('label', 'Latest News')->addPage($page);
        }
        $view->navigation($container);
    }

On thing that needs to be added is some form of caching (using Zend_Cache presumably ;)) otherwise this is going to be quite expensive with each page load.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Route requests for sitemap.xml to custom controller/action

Steven Lloyd Watkin — Wed, 06 Jan 2010 00:13:56 +0000

In order to direct requests for /sitemap.xml to a custom controller and action in your Zend Framework application simply add the following in your application.ini or alternative config file (e.g. I use navigation.ini):

resources.router.routes.sitemap.route                = "sitemap.xml"
resources.router.routes.sitemap.defaults.controller  = index
resources.router.routes.sitemap.defaults.action      = sitemap

Example code for outputting can be seen by creating an action in the appropriate controller (e.g. my sitemap lies in the index controller, sitemap action):

<php
class IndexController
    extends Zend_Controller_Action
{
    /**
     * Renders a sitemap based on Zend_Navigation setup
     */
    public function sitemapAction()
    {
    	echo $this->view->navigation()->sitemap();
    	$this->view->layout()->disableLayout();
    	$this->_helper->viewRenderer->setNoRender(true);
    }
}

Sitemaps can quickly and easily be generated using Zend_Navigation, a great quick tutorial (and generally very useful for Zend Framework tutorials) is Zend Casts – Dynamically creating a menu a sitemap and breadcrumbs.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Zend Framework Per-Module based settings

Steven Lloyd Watkin — Fri, 01 Jan 2010 22:40:16 +0000

I’ve created a followup to this post which requires less configuration, please see Module Based Layout – Zend Framework.

When using the zend framework with modules, its obvious that if you’re running various (sub-)sites off the same application you don’t necessarily want the same layout scripts for each part. I decided to go with the following site structure:

/Application
    /controllers
        ...
    /models
    /modules
        /default
            /controllers
            /layout
                /scripts
            /views
                /scripts
        /anotherModule
            ...
    /scripts

The problem was setting up the layout scripts on a per-module basis. The answer came through using an Action Helper. Setting up the layouts on a per-module basis involves three steps:

Application.ini (or similar configuration setup):

admin.resources.layout.layoutPath = APPLICATION_PATH "/modules/admin/layouts/scripts"
default.resources.layout.layoutPath = APPLICATION_PATH "/modules/default/layouts/scripts"
member.resources.layout.layoutPath = APPLICATION_PATH "/modules/member/layouts/scripts"
affiliate.resources.layout.layoutPath = APPLICATION_PATH "/modules/affiliate/layouts/scripts"

Create your Action Helper:


 * @since  2010-01-01
 */
class Pro_Controller_Action_Helper_SetLayoutPath
    extends Zend_Controller_Action_Helper_Abstract
{
    /**
     * Sets layout path based on module
     */
    public function preDispatch()
    {
    	$module = $this->getRequest()->getModuleName();

	    if ($bootstrap = $this->getActionController()
	                       ->getInvokeArg('bootstrap')) {

	        $config = $bootstrap->getOptions();

	        if (isset($config[$module]['resources']['layout']['layoutPath'])) {
	            $layoutPath =
	                 $config[$module]['resources']['layout']['layoutPath'];
	            $this->getActionController()
	                 ->getHelper('layout')
	                 ->setLayoutPath($layoutPath);
	        }
    	}
    }
}

And lastly boostrap the action helper:

...
    /**
     * Sets up layout scripts on a per-module basis
     */
    protected function _initLayoutHelper()
	{
	    $this->bootstrap('frontController');
	    $layout = Zend_Controller_Action_HelperBroker::addHelper(
	        new Pro_Controller_Action_Helper_SetLayoutPath());
	}
...

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Doctrine: DATETIME default NOW()

Steven Lloyd Watkin — Wed, 30 Dec 2009 18:30:04 +0000

I’ve been struggling with setting up a database schema for a new Zend Framework project. I’m ~~using~~ trying to use Doctrine ORM for my database models. I need to set up the schema so that it allowed me to set a default date and time for a `datetime` column, e.g. when adding a new message I get the current timestamp. After much searching and experimenting I found the solution so I’m sharing it.

In your schema YAML file simply do the following:

Message:
  actAs:
    Timestampable:
      created:
        name: created_at
        type: timestamp
        format: Y-m-d H:i:s
      updated:
        name: last_updated
        type: timestamp
        format: Y-m-d H:i:s
  columns:
    id:
      type: integer
      primary: true
      autoincrement: true
    name: string(255)
    email: string(300)
    message: string(2000)

If on the other hand you don’t want an `updated_at` column you can use the following:

Message:
  actAs:
    Timestampable:
      created:
        name: created_at
        type: timestamp
        format: Y-m-d H:i:s
      updated:
        disabled: true
  columns:
    id:
      type: integer
      primary: true
      autoincrement: true
    name: string(255)
    email: string(300)
    message: string(2000)

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

PHP Design Patterns – Observer Pattern

Steven Lloyd Watkin — Tue, 29 Dec 2009 22:02:38 +0000

I’ve been reading Head First Design Patterns recently and have decided to write some of the patterns as PHP examples for my own benefit. The first one that I’ve decided to code up is the Observer Pattern. The formal definition of the Observer Pattern is:

The observer pattern (a subset of the asynchronous publish/subscribe pattern) is a software design pattern in which an object, called the subject, maintains a list of its dependents, called observers, and notifies them automatically of any state changes, usually by calling one of their methods. It is mainly used to implement distributed event handling systems.

As systems become more loosely coupled making sure that when an event happens all systems that require knowledge of these updates are informed. For example, a blog post, after saving a post we may need to update a search engine (e.g. Lucene), update our sitemap, tags, email subscribed users, etc. The observer pattern allows developers to add additional listeners without editing their observable object. By injecting observers (i.e. a search engine update observer, a sitemap generator, etc) into a subject (i.e. blog post editing system) we can allow the it to perform all the necessary updates without any changes.

Before the Observer pattern was identified the usual trick would be to update the observable object with an additional line of code to update the required system, and removing the line of code as required. This does not allow for easily adding and removing observers.

The subject updates all of its observers via an update method. This update method calls a method in each observer which is derived from implementing an interface. Observers can add an remove themselves through methods in the observable object.

That’s basically it! I always find an example to be the best method of learning/understanding so here’s my coded up example…

Observer Pattern in PHP

In my example I’ve created a news system (NewsAggregator) which sends out news headline updates to smaller news feeds. Here the news sytem takes the place of the Subject, Observable, etc whereas the news feeds take the role of the Observers or Listeners.

Once initialised observers can attach and detach themselves from the subject as they see fit. In my example I have created three observers, these scan the headlines sent out by the subject and ‘shout’ the news if its appropriate. The three observers are named below along with the terms they scan for when receiving news headlines:

Sport Observer: ‘rugby’, ‘football’, ‘tennis’
News Observer: ‘politics’, ‘finance’, ‘government’
Gossip Observer: ‘celebrity’, ‘music’, ‘fashion’

After initalising the subject I add the news and gossip observers and send out a news update. After this the sport observer is added before more news updates are sent out. Lastly the gossip observer is removed before a final news headline is sent out.

The three different observer classes implement the interface Observer, this gives them a clear interface/method through which they will receive updates. Provided they implement the Observer interface they will be able to attach themselves to the Subject. This also keeps with the programming paradigm of ‘program to interfaces not implementations’. The NewsAggregator class extends the abstract class Subject, which provides us with the three required public methods:

updateObservers()
addObserver()
removeObserver()

The code can be seen running here, Observer Pattern in PHP Running, and the code can be downloaded from here, Observer Pattern in PHP Code.

Observer Script

_observerList[] = $observer;
	}

	/**
	 * Method to remove an observer
	 *
	 * @var Observer $observer
	 * @return boolean
	 */
	public function removeObserver(Observer $observer)
	{
		foreach ($this->_observerList AS $key => $ob) {
			if ($ob == $observer) {
				unset($this->_observerList[$key]);
				return true;
			}
		}
		return false;
	}

	/**
	 * Method to update observers
	 *
	 * @var string $newsHeadline
	 * @return void
	 */
	public function updateObservers( $newsHeadline )
	{
		foreach ($this->_observerList AS $ob) {
			$ob->update( $newsHeadline );
		}
	}

	/**
	 * Add a new news story
	 *
	 * @var string $story
	 * @return void
	 */
	public function addNewsStory( $story )
	{
		if ( empty( $story ) || !is_string( $story) ) {
			throw new InvalidArgumentException('Expected a news story!');
		}
		$this->updateObservers( $story );
	}
}

Subject / Observable Script

_observerList[] = $observer;
	}

	/**
	 * Method to remove an observer
	 *
	 * @var Observer $observer
	 * @return boolean
	 */
	public function removeObserver(Observer $observer)
	{
		foreach ($this->_observerList AS $key => $ob) {
			if ($ob == $observer) {
				unset($this->_observerList[$key]);
				return true;
			}
		}
		return false;
	}

	/**
	 * Method to update observers
	 *
	 * @var string $newsHeadline
	 * @return void
	 */
	public function updateObservers( $newsHeadline )
	{
		foreach ($this->_observerList AS $ob) {
			$ob->update( $newsHeadline );
		}
	}

	/**
	 * Add a new news story
	 *
	 * @var string $story
	 * @return void
	 */
	public function addNewsStory( $story )
	{
		if ( empty( $story ) || !is_string( $story) ) {
			throw new InvalidArgumentException('Expected a news story!');
		}
		$this->updateObservers( $story );
	}
}

Controller Script

';
}

// What are we doing?
echo 'Observer Pattern Example in PHP' . PHP_EOL;
echo '================================' . PHP_EOL;
// Set up our subject
$subject = new ArticleAggregator();
echo ' - ArticleAggregator created' . PHP_EOL;

// Add some observers
$subject->addObserver( new NewsObserver() );
$subject->addObserver( $gossiper = new GossipObserver() );

echo ' - Added NewsObverser & GossipObserver' .
	 PHP_EOL . PHP_EOL;

// Beep, beep, beep... News Flash!
echo 'NewsFlash: celebrity rugby player loves finance' . PHP_EOL;
echo '================================================' . PHP_EOL;
$subject->addNewsStory('celebrity rugby player loves finance');
echo PHP_EOL;

echo ' - SportObserver has found out and wants to join the group!';
$subject->addObserver( new SportObserver() );
echo PHP_EOL . PHP_EOL;

// Beep, beep, beep... News Flash!
echo 'NewsFlash: government messes up again!' . PHP_EOL;
echo '=======================================' . PHP_EOL;
$subject->addNewsStory('government messes up again!');
echo PHP_EOL;

// Beep, beep, beep... News Flash!
echo 'NewsFlash: fashion and football combine' . PHP_EOL;
echo '=======================================' . PHP_EOL;
$subject->addNewsStory('fashion and football combine');
echo PHP_EOL;

// Beep, beep, beep... News Flash!
echo 'NewsFlash: music and politics, what next?' . PHP_EOL;
echo '==========================================' . PHP_EOL;
$subject->addNewsStory('music and politics, what next?');
echo PHP_EOL;

/**
 * Gossipers grow tired of news very quickly and have decided
 * to stop listening, despite all the interesting news today!
 */
echo ' - GossipObserver is bored and leaves the group!' .
     PHP_EOL . PHP_EOL;
$subject->removeObserver( $gossiper );

// Beep, beep, beep... News Flash - Update to an earlier story!
echo 'NewsUpdate: fashion and football combine says ' .
     'government' . PHP_EOL;
echo '================================================' .
     '=========' . PHP_EOL;
$subject->addNewsStory( 'fashion and football combine ' .
                        'says government' );
echo PHP_EOL;

if (!empty($_SERVER['HTTP_USER_AGENT'])) {
    echo '

}

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Office Grid Computing using Virtual environments – Part 4

Steven Lloyd Watkin — Fri, 04 Dec 2009 23:59:53 +0000

Introduction

I work in a company where we run many batch jobs processing millions of records of data each day and I’ve been thinking recently about all the machines that sit around each and every day doing nothing for several hours. Wouldn’t it be good if we could use those machines to bolster the processing power of our systems? In this set of articles I’m going to look at the potential benefits of employing an office grid using virtualised environments.

In part 3 we created our virtual processing machine and set up windows machines to become idle-time workers.

Running the latest code

Inevitably after creating your workers business logic will change, bugs will be found, faster more efficient code will be produced thus leaving your workers sat around processing data using old smelly code. How then do we ensure that we’re always using the latest and greatest version of our processing scripts?

There are a few very easy simple ways we could do this, the trick, however, is to reduce processing power and network traffic in achieving this. Lets start with the simplest of solutions and improve it slowly over a couple of iterations.

The first method would be to simply connect to our job control server (via samba, FTP, or similar) and pull down the latest version of the code. Not very efficient, but it will do the job. Lets improve on that somewhat, how about creating an rsync script and using that each time instead? Alternatively what about putting our latest processing script into subversion checking out the code initially and then just updating our code on each run (svn update)?

In the end we could end up with a bash script (called by cron every 10 minutes) which looks as simple as this:

#!/bin/sh
if ps ax | grep -v grep | grep php > /dev/null
then
    echo "Job is currently processing, exit"
else
    echo "Job is not running, start now"
    cd /path/to/working/copy
    svn update
    php yourJobProcessingScript.php
fi

Now we can be sure that with each run we’re definitely running the latest code. We’re ensuring this by updating our code base each and every time we perform a run and reducing network traffic by only transferring the file differences across our network.

In my demonstration setup, I did exactly as above. Subversion was installed on my job processing server and I simply pulled the latest code from a ‘worker’ branch using ‘svn update’. I also added a version number tag to my processing script which was returned to the database as part of the results return. This way I could see that my code was being updated each time I copied my trunk into the worker branch i.e. that I was definitely running the latest processing script.

Using the latest data

If your job processing makes use of data sources then at some point these are going to be updated too. Unless you call your data sources on a very infrequent basis you’re going to flood your network with traffic as soon as your workers start running bringing everything to a standstill. For my solution I decided that I’d like to move my data sources around with my VMs.

Hold you’re horses there! What if my data sources are HUGE? Well this really is a case of how much data are we talking? It may be more cost effective to install an additional larger hard drive into each machine than to purchase an additional processing server. This is a question of budget and is up to the business to decide. It maybe that your data sources are so large that its just unfeasible to keep that amount of data in your worker machines. In that case what would you do? Well we could look at calling a local data server, but this might cause issues with the network. In this case a grid system such as this may become unrealistic to include in your office environment. It may also be that you can look into alternative running strategies, for example only calling your workers between 8pm and 6am each night and/or throttling data source requests.

Moving on lets say our data sources amount to 100Gb of data. Well yes that’s quite a bit of data to move around the network on an update. How would we ensure that we have the latest copy of the data in this case? Rsync is a possibility, but personally I think by running your latest data source on your job processing server and setting this up as a master in replication (with a nice long bin log) might be the way to go:

By setting each of your workers up as a slave to the job control server updates to your data sources will trickle down nicely to your workers without a huge increase in network activity (that is unless you perform a huge data update and all your workers kick in at once). This has advantages over rsync in that you wouldn’t get a long pause before each job; as the database updates, the mysql daemon on your worker will continually update its data while the processing continues.

This is how I set up my demonstration server. To set up replication I followed the guide on the mySQL site (Setting up replication) and within 20 minutes I had my inital worker replicating the job control servers dataset. For each additional worker the replication settings and process worked each time when the VM was copied.

Summary

In this section of the article we have looked at how easy and painless it is to keep your processing code up to date by usingÂ rsync or subverion (SVN) to do the work and reduce network traffic at the same time.Â We also discussed how to keep your data source information up-to-date by allowing it to trickle down to each of your workers. Thus we areÂ ensuring that we keep up with business logic and information in our office grid system. There will obviously be countless alternatives to performing these tasks, but here were two simple examples to show how easy a solution is to come by.

Next time

In the final part of this series, aptly named Part 5 , we’ll discuss deploying this system for. I’ll summarise what has been learned and what I managed to create.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Office Grid Computing using Virtual environments – Part 3

Steven Lloyd Watkin — Fri, 04 Dec 2009 23:37:55 +0000

Introduction

In part 2 we looked at the jobs a server will run, and how jobs should be configured in order to achieve greatest amount of processing whilst ensuring that each job is processed without fail.

Setting up your worker – or LiMP server

The next step in the process is to set up your virtual workers. For this I’m going to use an installation of centOS using VirtualBox. I’m going to install mySQL and PHP on the server, also known as a LiMP (Linux, mySQL, PHP) ServerÂ (I may have made that name up).

Install VirtualBox on your windows machine (follow link)
Download and install centOS (current version 5.3) within a created virtual machine

There’s no point me going to this there’s probably 1,000′s of great tutorials out there (ok, here’s one: Creating and ManagingÂ centOS virtual machine under virtualbox). The important point to note I suppose is that I called my virtual machine GridMachine.

As far as my choices of virtualisation client and operating system go there is no big compelling reason for each choice. VirtualBox is something I use on my home machine and is supported by the three major operating systems. I chose centOS as its a good stable OS and I use it on my own web server. I am a great believer in the right tools for the job (although I’m applying ‘use the quickest and easiest for you’ mentality here), so if operating system X runs your code quicker and more efficiently use that instead :)

Importantly make sure that your VM uses DHCP, otherwise for each new virtual machine would need to be configured separately which is something we don’t want.By using DHCP we don’t need to configure network settings individually for worker machines, DHCP will hand out IPs for you. Therefore you can copy your virtual machine about the office without worrying about setting each one up (this improves scalability and reduces worker administration).

The process you should aim to achieve would be to obtain a new physical machine, install VirtualBox, and then pretty much deploy the virtual image without much else. It might be wise to setup all your workers on a different subnet so that you can at least see how many machines are running. You’ll also need to set up your machines on a long lease or unlimited lease DHCP.

How to run Jobs on the worker

This is an interesting area and there are several valid methods for processing jobs on the worker. Here I’ll just discuss the two most obvious:

Perpetually running script: A script, be it a shell script, or a PHP script is executed once on the worker and runs as part of an infinite loop. I’ve discounted this method as one crash of the script and potentially your workers will cease to run without some sort of intervention.
Cron based script execution: Every X minutes the cron daemon kicks off a call to your script to get things going. Without some checking this could lead to many many copies of your worker script running.

My decision was to go with cron which kicks off a shell script every 10 minutes.Â My shell script performs the following tasks:

Get a process list and grep this for ‘php’. If not found then continue.
Call your job code, in my case this would be something PHP based
Worker script completes its run
Ready to go again on the next appropriate call

My bash script looks something like the following:

#!/bin/sh
if ps ax | grep -v grep | grep php > /dev/null
then
    echo "Job is currently processing, exit"
else
    echo "Job is not running, start now"
    php yourJobProcessingScript.php
fi

Note: the echo’s are almost completely pointless, but may help the next person who comes along to try and edit them.

That concludes the set up of the worker virtual machine, quick, simple, and easy to copy to each new piece of hardware that is received. The ‘cleverness’ of the grid system really isn’t in the visualised OS, its all to do with the code created to process jobs, the job configuration, and in making sure that the job runs when appropriate (i.e. when the host is idle).

Setting up Windows to Initialise Workers

The first task is to work out the command required to run the virtual machine from the windows command line. If you’ve installed virtualBox in the default location and you’ve named your worker GridMachine then the command required to load up your worker is:

"C:\Program Files\Sun\VirtualBox\VBoxManage.exe" startvm GridMachine

However to run the script in a ‘headless’ state we need to use:

"C:\Program Files\Sun\VirtualBox\VBoxHeadless.exe" -startvm GridMachine --vrdp=off

This will start the virtual machine without the GUI and allow it to save state gracefully. The second argument turns off RDP so it doesn’t conflict with windows RDP, or give you a message about listening on port 3389. The virtual machine name is cAsE sEnSiTiVe!

Next, we’ll need to set windows up to kick off our worker VM once the machine has been idle. To do this (on Windows XP) you’ll need to go Start -> All Programs -> Accessories -> System Tools -> Scheduled Tasks as below:

Next click on ‘Add Scheduled Task’ followed by browse to add a custom program. Navigate to your VBoxManage script and click ok. Schedule your task for any of the options (we’ll change this in a minute) and continue. After skipping the next screen windows will ask you who you want to run this task, I’d suggest either ‘Administrator’ or creating a new privileged user. Remember we don’t want to interfere with the standard staff account on the machine at any point. Click next and check show advanced options for this task.

To the end of the run textbox add our ‘startvm GridMachine‘ string and ensure that run only when logged in is left unticked. Visit the schedule task next and change the schedule drop down to the option ‘when idle’, choose the amount of time you’d like the machine to be idle before moving on to the next tab.

Finally untick the option which states stop the task if it has been running X amount of time, but do tick the option to stop the task if the machine is no longer idle.

That’s it then for the windows host setup!

Summary

In this part we have set up a virtual machine to act as a worker, as well as the way in which we call and execute our job processing scripts (for myself a PHP script). From here we look at how to set up our copies of windows to start up the virtual machine in headless mode when the computer becomes idle, and save its state when the user resumes usage of the machine. Hopefully at this point you’re seeing how simple it is to set up such a system and are itching to get some experiments going yourself!

Next time

In Part 4 we’ll be looking at using tools to ensure that you’re running the latest version of the code and data sources so that obtained results are always up-to-date with the latest business information and logic.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Office Grid Computing using Virtual environments – Part 1

Steven Lloyd Watkin — Fri, 04 Dec 2009 23:23:57 +0000

Introduction

As a PHP developer I’m going to use tools that I use each day namely, Linux, mySQL, PHP, VirtualBox and subversion (SVN). However I hope this guide will adapt to other languages and technologies just as well.

The solution I provide will be very loosely based on the type of processing we’d need to achieve however this may not be true through the entire article as I’ll change things for simplicity, or to produce more interesting usage scenarios.

These virtualised environments will run on windows machines since this is what the majority of offices run. The processing that the office machines do should not interfere with staff using those machines, should require no maintenance at the machine, and be easily deployable to new machines as they become available. Also, new virtual machines should not require any additional configuration as this greatly reduces the scalability and ease at which the grid system can be extended.

Why Deploy an Office Computing Grid?

Firstly you may be thinking,why not just use a cloud computing resource such as Amazon’s EC2 platform? Well the reasons could be several, for example:

You won’t entrust certain data to a cloud computing environment
You can’t put certain data into a cloud computing environment for legal reasons (e.g. data leaving the country), potentially for legal reasons, e.g. NHS records.
You want to keep your processing units close and have full control over the hardware too
You don’t have the project funds to run cloud instances
Your office doesn’t have a connection to the internet and therefore its not possible to use a cloud resource
You don’t like rain, clouds suggest rain, therefore you keep well away

I’m sure the list could continue, but I think that’s enough for now.

Advantages of an Office Computing Grid

Well, lets do some maths (and in true physics style lets make some sweeping assumptions). Imagine you have big beefy processing server running 100 jobs per day. In your office you have 50 machines which are idle 16 hours a day, each of these machines is 10% as powerful as your beefy processing sever. (All results here are rounded to underestimate performance increase).

So, 1 machine * 10% power * 2/3 time = 0.067 i.e. 1 desktop processing in idle time could process 6 full jobs per day.

If you now scale this up it takes 15 idle desktops to process as many jobs per day as your main processing server does.

So in our pretend office of 50 machines we could increase our processing power from 1 server up to 4 full processing servers, or we could be processing 400 jobs per day instead of 100.

Notice, for no investment in new hardware your company has just increased its batch processing capacity 4 times! Potentially you’re going to increase your power usage but from most office environments I’ve been to machines are generally left on overnight anyway, so you could see this as a green initiative.

Other advantages also mean that investment in new (or updated) processing servers can be delayed if your office machines are sufficient and that as you improve the power of your office machines your office grid becomes more powerful automatically.

Technologies

What you need? (or more correctly what did I use):

Idle office machines (in my case a spare old windows XP laptop)
VirtualBox (or another virtualisation client software)
A virtual machine with PHP, mySQL runningÂ running a cut down OS, I’m calling these my LiMP servers :)
Jobs to run
Job server (can be another virtual machine somewhere)

Typical Jobs

The types of jobs that this system is designed to run is as follows:

System receives a list of data upon which we need to match and return results
Matching involves checking/searching several (fairly static) data sources
Results from data sources may require further validation, merging, checking of additional data sources in response to results
Data is returned with matching records, fully validated and processed
Each record within a job is independent of the rest

So basically we’re looking at running jobs which require a mixture of database lookups and some number crunching, a fairly typical scenario in a business environment.

Grid solutions are not only advantageous for processing jobs of this type. Basically, any process which can be split into independent units can be run in parallel. See this wikipedia for examples and more information: Grid Computing, but a couple of famous examples are Seti@Home and BIONC. There are frameworks for running computing grids, and these are well worth looking into.

What will we achieve?

By the end of these articles I hope to show that deploying an office grid need not be hugely expensive or time consuming. I’m going to discuss:

Setting up the job control system, job configuration
Creating an appropriate processing virtual machine
How to setup the system on a windows machine
Ensuring you are using the latest code and data
Deployment and benchmarking
Looking ahead

I’ll be building (ok I built, then wrote this) an example application to test the concepts on a local machine using windows XP and my ‘GridMachine’ virtual machine. My job control server will be my main machine which runs Fedora 11.

This is in no way meant to demonstrate a fully working robust system, its meant more of a demonstration and discussing showing that these things can be achieved in a reasonably short space of time and at little cost. Please feel free to send me any comments, corrections, or improvements and I’ll do my best to keep this article updated to match.

Next time

In part 2 I will start by looking at the job control system, and look into how jobs should be configured in order to achieve greatest amount of processing whilst ensuring that each job is processed without fail.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Office Grid Computing using Virtual environments – Part 2

Steven Lloyd Watkin — Fri, 04 Dec 2009 23:23:00 +0000

Introduction

In Part 1 I gave an overview of the system and technologies I will be using as well as discussed some of the potential reasons why you would want to create an office grid.

Job Control

If you’re going to be running jobs then you’re going to need some way to manage them. Your job control system (on your job server) needs to be really well thought out before even attempting to run an office grid. So firstly, what are the tasks for a job control system:

Hand out jobs upon request from workers
Tell workers what type of jobs to run
Track jobs
Ensure that jobs are only run once
Provide job data to workers, or at least tell them where to get it

The system also needs to be extensible, a solution that works for now in a single case may be extended to run several types of jobs as the business sees the worth in a grid solution. For example, jobs may gain priorities, more than one job type may exist (i.e. several code bases), eventually you may even run several different worker machines that are optimised for each type of job (although that does move away from the ‘generic worker’ idea). Always try to think about the future when developing systems, a short term vision can lead to longer term frustration and increased development time.

Job Server

We’re going to need somewhere to control our jobs from, this should be the only system in your grid that has a fixed resource locator, be that an IP address, host name, URL (using internal DNS), etc. This is because the workers need to know where to look for jobs, workers need to find the job control system (not the job control system find the workers).

The job server itself doesn’t really have a complicated task (in a basic system anyhow), it needs to store a list of jobs, hand out jobs, receive results, and subsequently store them for later retrieval. How these parts (such as ‘hand out jobs’) are defined can be very basic. Later on we can extend the system to include an administration interface to add, edit, delete, suspend jobs but this is beyond this exercise.

There is no reason whatsoever then that your job server could not be a virtual machine running within your main processing server provided it doesn’t drain too many resources from it. The job server however does need high availability, if it goes down on a Friday evening you’re going to lose a whole weekend of processing, potentially costing you a couple of weeks worth of processing time (when compared to your main processing server alone). You may want to consider putting your job server on a load balanced environment for high availability.

Basic Setup

The basic setup for our job server will consist of what I’m calling one of my LiMP servers (that is Linux, mySql, PHP). The code running on theÂ workers will actually work out what jobs it can run by interacting with with job control system databases. Later on we could create a web service and actually hand out jobs rather than having the workers do the hard work themselves, but for now we’ll continue using the KISS principle (Keep it Simple, Stupid!).

So, lets create three mySQL tables to deal with jobs. These will be `jobs`, `jobRecords`, and `jobResults`.

Here I’m using SQL Buddy a great little alternative to phpMyAdmin just because its easier to install on centOS (for others see: 10 Great alternatives to phpMyAdmin)

This table consists of 5 simple fields,

id: Uniquely identify the job
name: Could be a client reference, or any number of other identifiers
Status: You need to know where the job is at, e.g.
- 0: Not started
- 1: Picked up
- 2: Completed
started_by: Who’s started doing the job? This isn’t entirely required but is a nice to have. I’d suggest tracking workers by their IP address on your network
started_at: When did the worker start the job? By tracking jobs that have not completed within X amount of time we know we need to pick up the job once again and start processing by another worker. Workers could stop processing/go offline for any number of reasons, power failure, crash, network loss, etc.

It is easy how this table could be extended with a few additional fields to allow for statistics tracking, a finish time column to see how long the job took, a counter to see how many workers picked up the job (obviously this needs to tend to 1), job priority, the list can go on and on. In more complex job scenarios it would be possible to specify how much memory the worker would need access to (and therefore only use suitable workers), or even what type of worker would be required.

Lets add a few example jobs:

The next table again is quite simple to understand, these are our job records. They are linked to the main jobs table by a column `jobs_id`. The make up of this table very much depends on the data that you need to supply to your workers, lets make a very simple example where we have four columns:

id: ID of the record
name: Person’s name
address: Person’s address
jobs_id: The job ID that this record is linked to

The third and final table consists of a results table, it has much the same make up as our records table, and with the addition of some columns could be part of the records table:

job_record_id: Link the result to the job table
result: The result data

…and that’s all you need for job control! (albeit at a very basic level) In my case I’m pointed to another table where my data to process was located, but this could just as easily been a file, parameters to run simulation code, you name it.

Selecting a job

As stated previously, the workers will do our job management for us for now, so all we need to really do is find a job that needs processing and get the information. How would we do this? Well pick our job selection criteria and look for jobs, in SQL I did the following:

Take any jobs that are not marked as complete but from our worker and reset them (substitute __ME__ with an identifier, easiest would be IP address):
```
UPDATE `jobs` SET `status` = 0 WHERE `status` = 1 AND `started_by` = __ME__;
```
Using our job selection criteria, select a job and tell the control system that this worker is dealing with it:
```
UPDATE `jobs` SET `status` = 1, `started_by` = __ME__, `started_at` = NOW() WHERE `status` = 0 OR
(`status` = 1 AND `started_at` > DATE_SUB(NOW(), INTERVAL X HOUR)) ORDER BY `id` ASC;
```
By grabbing jobs that haven’t returned results in X amount of time we ensure that all jobs are run in the event of a worker crashing or going AWOL.

Next grab the jobs details followed by the records themselves:

SELECT * FROM `jobs` WHERE `started_by` = __ME__  LIMIT 1;
SELECT * FROM `job_records` WHERE `id` = __JOBID__;

Upon completion of the job we insert our result records and mark the job as complete. Remember as jobs can suspend/resume at any time allow for some robustness in your script. It might be that the task suspends half way through updating the job control system, so checking the number of records in a job and the number of results saved back to the job control system would be a wise move.

In addition, whilst this demonstrates how jobs can be selected and managed from an SQL-query frame you should really be abstracting your job control so that if you decide to switch to using a web service, a file based system, XML, or any other number of systems it will not affect the code above it.

Job Configuration

The next aspect to consider is job size and configuration. By playing with job configuration we can strike an excellent balance between speed, process replication, and reliability. Take a couple ofÂ scenarios:

Jobs take 1 day each to run: This means that your workers need 15 days to process each job (remember 10% of the power for 2/3rds of the time). This is clearly not a wise configuration, your job size is way too big! It would take at least double the time to get a job processed should the initial worker go AWOL (time to pick up that it hasn’t returned a result plus reprocessing time). In an ideal you’d have at least one full job easily cleared by the end of each long idle period, that way you keep the jobs ticking over and at worst case a job would take two days to process should the first go missing.
Jobs take 1 minute to run: This means that your workers take about 15 minutes to run each job. Whilst this may initially seem ideal, you gain additional work processing during lunch time, coffee breaks, meetings, etc this scenario puts strain on other areas of your system and introduces its own problems. For example, firstly your setup/processing time ratio is going to go right down, therefore losing system efficiency. Your network is going to be constantly streaming job information to the various workers frustrating staff who are dong their day to day work. You’re also going to put more strain on your job processing server as it has to dish out lots and lots of small pieces of work on a regular basis. Lastly, in this situation if your job server goes down you’re going to create a huge back log of uncompleted work whereas bigger jobs could of continued processing blissfully unaware that the job server was experiencing difficulties.

In reality there will be no one ideal configuration for your grid setup, much depends on the available resources, types of job, job turnaround time requirements, network capability, and so on. However some guidelines would be:

Size jobs so that each worker can get through at least 3-4 jobs in a period of 15 hours (the longest likely idle time period)
Play with the job size so that setup time becomes fairly insignificant compared to the processing time (bearing in mind the above point).
If a job doesn’t complete in double the amount of time (maybe less) you expect it to complete it assume that its gone AWOL and start processing it with another worker. This means you may have to wait up to three times the normal length of a job for it to complete (possibly longer if the subsequent job fails). You may want to reduce this time, but be careful not to reduce it too much as you may start duplicating processing tasks on a regular basis.
Jobs should be independent of outside requirements as much as possible. The job server, for example, should only be contacted at the start and end of every job.
Don’t saturate your network, this will have two negative effects, your daytime staff will find using the network frustrating and problems may be experienced with connections timing out a problem that will only get worse as you scale your grid.
Ensure jobs can run on your workers. If jobs become too memory intensive or disk space intensive jobs will start aborting and the only thing you’ll notice is a drop in number of jobs processed with no real reason why.

Submitting Results of a Job

When submitting the results of a job it is important to check that results have not been submitted by another worker, especially if the current worker has been dormant for some time.

When results are submitted ensure that the number of results matches the number of records within the job.

As stated previously, and can not be over emphasised, build fault tolerance into job retrieval and results submission. The workers can (and most likely will) go into suspend mode at the most inconvenient of times and this needs to be catered for. Also once again abstracting away your results submission will help cater for future changes to your job control system much easier to deal with.

Summary

In this sectionÂ we have looked at what a job control server needs to do and how to get a very basic system set up. We discussed how to retrieve a job from the control system and how best to configure jobs to get the most our of your office grid system. To finish, a paragraph or two on submitting results back to the job control server was presented.

A job control server manages jobs and ensures that all work units are completed
By abstracting your job select/results submission we can change the technology of the control server without much problems
Configure your jobs to ensure that they are run quickly and efficiently without putting too much pressure on your network infrastructure, and without duplicating processing tasks on a regular basis.
Ensure that you build fault tolerance and error checkingÂ into your routines, workers can suspend and resume and the most inconvenient of times. Remember to check if results have already been submitted by another worker.

Next time

In part 3 we’ll create our virtual processing machine and set up our windows machines to become idle-time workers.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Office Grid Computing using Virtual environments – Part 5

Steven Lloyd Watkin — Fri, 04 Dec 2009 23:03:08 +0000

Introduction

In Part 4 we looked at using tools to ensure that we’re running the latest version of the code and data sources so that obtained results are always up-to-date with the latest business information and logic.

Pre-Deployment

Before deploying your grid system if there’s one thing you do and one thing alone it’s benchmark your current system! No matter what you tell colleagues about how much extra work your system is going to do unless you have numbers to back this up your guarantees are nothing. So,

how many records can you process currently? Per Day? Per Hour?
How long does it typically take to turn around a job?
How much more capacity do you have?

There’s also additional questions:

If your processing server (or one of your processing servers) goes down how will this affect your capabilities, will you be crippled?
What advantages do you hope/expect to get from a grid system?
Are your office machines capable of running the jobs?
Are your (or can you jobs be converted) to wrok in this style of running?

The last major point is to take your time on any major change like this. Update your processing code to work using the new methodology, benchmark again. Possibly set up your processing server to run a virtual machine, after all your processing server will just be another worker (just a very powerful one relatively). Allow the new process to settle.

Deployment

My suggestion would be to pop into the office one weekend perform all the installations and setup. Do this just before a fortnight’s holiday and leave so other poor chap to deal with the consequences… maybe not…

Deployment for a system like this needs to be slow. Despite it being relatively simple to set up this system will affect your entire office infrastructure (well the digital one). Firstly, roll out to a couple of machines at a time, monitor network traffic, how the worker hosts perform on a day-to-day basis. You may need to alter your job configuration in response to your findings.

Once the system has settled with a few machines (lets say 10% of all office machines, i.e. 5) keep monitoring network traffic and host machine performance.Â Next benchmark again, you should now be processing 33% more jobs than your first benchmarks. Check this is so, or that you’re at least in this ballpark. If not, investigate what is going on before moving on. Repeat this cycle until you happily have all office machines running without killing individual machine performance or grinding your network to a standstill.

At all times keep benchmarking, even after all deployments are made. Check how new code updates affect speed of your system, check all workers are reporting in and processing jobs. Slowly (very slowly) increment your job configuration to get the best from your workers and network.

Stop!

What if you want to stop your workers from running at some time? They are all out there running, regenerating, and trying their best to process data like hungry insects. The answer may seem obvious but its worth adding just in case its overlooked. Simply edit your processing script with an exit(0) or die() or some other statement to kill your processing job. An important reason why we always try to update to the latest processing script before any run!

Demonstration System

In order to write this set of short articles I created a very small grid to demonstrate the technologies and methodologies. I read lots of articles, tutorials, and used various tools to setup and monitor what was going on. By no means have I gone out and saturated a whole office with traffic and nor have I had access to a regular staff members PC to see how host performance was affected.

My demonstration system was very humble indeed. I used my regular desktop set up as a job control server. On this I had installed mySQL server installed set up as a master in replication, PHP,Â and SVN linked through apache (for access via worker VM).

I then created a centOS worker machine on VirtualBox on a 6 year old windows XP laptop. I setup scheduled tasks as specified after copying the VM onto the machine and let it go.

The virtual machine was set up with PHP, subversion, and mySQL. I checked out a branch named ‘worker’ from my job control servers repository and made sure it could be updated using ‘svn update’. Next I setup mySQL as a slave and checked that data was replicating from mySQL on the job control server down to the worker VM. After all this I setup the bash script and the cron job.

My processing script basically went along the lines of this (very simple stuff):

Read in the name field
Counted the number of similar names in a table from the data source held on the VM
Counted the number of names as above but splitting the name by spaces (i.e. forename, middle, surname)
Repeated this process 1,000 times

Each job took approximately 20 minutes to run. At one point I opened several copies of the worker VM on the windows laptop and watched the jobs be checked off by each of the worker IP addresses. At this point I also confirmed that replication automatically restarted.

Leaving the laptop to idle resulted in a worker starting to process jobs from the job control server. When resuming laptop usage there was a delay of about 30-60 seconds, this is a fair amount of time and staff would need to be made aware that their machine may pause for a short while when returning to the machine. Newer machines may not have a pause of this long. The benefit of the amount of processing performed by these machines during idle periods would more that outweigh staff members having to wait a short period (say 1 minute) on arriving at their machines of a morning (I frequently wait longer that this for a Windows Defender update to take place) provided they were made aware of this (useful time to grab a morning coffee!).

Overall I feel confident that I have demonstrated the technologies that could be used to create such a system. I have shown that such a system does work on a (very) small scale and with some more experimenting could be scaled up utilise the resources of an office’s machines. If I don’t get to the point of doing this I would be very interested to know/see when someone else does.

Conclusions / Evaluation

The next obvious step would be to actually get a real world example and start to deploy a system such as this within an office environment and see what happens. Asking a business to commit to this without a trail blazing company to prove the technology and effectiveness may be a little difficult. Grid/Distributed computing is very popular is some circles and has some large applications (BIONC, SETI@Home, Folding@Home, etc). I did not, however, find a smaller scale and simple system like this in my searches that could be rolled out within an office environment.

I created a basically free system using mostly open source software and tools available in almost any office. The technologies were basically demonstrated and show to perform and work as expected. Hopefully I have show that with not much work and with a very simple setup you can deploy an office grid computing system that is powerful, cheap,Â and scalable all at the same time.

Once a system is up and running there is almost no end to the amount of customisation and improvements you can make. For example statistics / benchmarking can easily be added showing the worth of such a system every day. New machines can be added quickly and easily as and when they arrive with upgrades to existing hardware bolstering your processing power.

I hope you’ve enjoyed reading this series of articles and its given you food for thought on running an office grid system. The solution presented here won’t necessarily work in all situations but should be adaptable to allow you to get your data processing done using your own solution.

Please feel free to send me any comments, corrections, or improvements and I’ll do my best to keep this article updated to match.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Zend Framework: Fundamentals – Review

Steven Lloyd Watkin — Sat, 28 Nov 2009 22:42:27 +0000

My employer recently paid for a group of us developers to take the Zend Framework: Fundamentals course, here I’ll summarise my thoughts and opinions on the course for others. For those looking to save time, here’s my summary:

For developers who haven’t had time to look at the Zend Framework this course (Zend Framework: Fundamentals) offers a good overall picture of the framework introducing you to the key areas and giving enough information in order to continue. For those who have spent time looking at the framework and have followed one or two tutorials this course does not offer much beyond.

Background

I’ve been a PHP developer for around 5-6 years, and have started working with the Zend Framework on a component basis over the last 6 months. I’ve developed and/or been a developer on a couple of small Zend Framework MVC sites.Â I’ll be honest, I haven’t had a huge amount of exposure to other frameworks from a coding point of view but have spent several hours researching the project websites and evaluating them.Â The framework and the community surrounding Zend Framework it is quite exciting and there seem to be huge possibilities in where its going.

About the Course

The course is delivered over 9 two hour webex sessions (with a 10-minute break in the middle). The time is spent going through a set of slides provided by Zend with discussion at any time. You can use a microphone to talk to the instructor, but to be honest I didn’t see anyone use anything more than the chat window. In addition a VMWare Ubuntu machine is provided that has example code and projects set up an a trial version of Zend Studio. The course leader talks to attendees either over an integrated VoIP solution, or you can dial in using one of the many worldwide dial in numbers.

During the course the material consists of a brief overview of the Framework and the MVC pattern before heading into a sample guestbook application. The discussion demonstrated bootstrapping, Zend_Application, Db Tables, Database access, Forms, Filtering, ACL, Validating, etc, etc. Basically covering all the topics you’d require to get a basic site up an running all the time giving you the tools to go and get more advanced in the framework (although this did amount to ‘See the website’ much of the time).

Time is given to code up some examples, and to develop the ‘guestbook’ and simple ‘wiki’ application. Personally I felt that providing the code or each app and then asking us to develop what was essentially a copy alongside didn’t really provide a good learning experience. I would have preferred to develop an application similar, but not identical. to the example application with the benefit of having a guide to refer to. Alternatively building the applications from scratch with the demonstrator would of possibly led to more questions about why and how, thus giving a better understanding of the framework, after all you can look up specifics after the course.

The last lecture consisted of working on the wiki application with help/guidance from the instructor. After the course feedback was taken, it was emphasised several times through the course that Zend takes feedback very seriously, in fact apparently our version of the course was quite new. Some of the other developers in the company will be taking the course soon so it will be interesting to see if this has happened.

The course style was informal, allowed for feedback and collaboration between attendees and the instructor. The course leader was friendly, approachable (email addresses were shared for questions), and whilst his presentation from the slides was a bit shaky seemed fully competent in the framework. He was clearly someone who used the framework on a regular basis rather than someone who is taught to teach the course, I liked the ‘real world’ experience in that respect.

Overall Feeling

In some ways I found the course a waste of time, in others it was very handy. Hopefully I’ll get my reasons across clearly, and maybe provide some food for thought or useful feedback (knowing me this is unlikely!).

For myself this course was aimed at too low a level. Having gone through the quickstart guide, read Rob Allen’s Zend Framework in Action, and worked with the framework a little I didn’t really get anything too much. I would of liked the course to pick up from the end of the quickstart and develop additional skills.

That said, the course title does clearly state “Zend Framework: Fundamentals” and in that aspect the course achieves what it sets out to do. Other members of the development team that haven’t spent the time looking into the framework finished each session with enthusiasm and asked questions which was really nice to see.

All was not lost, it was good to spend time confirming the basic details of the framework and get to ask a couple of questions in areas where I wasn’t 100%. It was also time that I got to sit down each day and think about coding using the framework and future projects, something I wouldn’t of been able to do otherwise (can you imagine your company agreeing to that? :) ). Last but not least you also get a nice certificate from Zend to say that you attended the course (albeit by email).

Zend Framework Certification

This was one question that kept coming to mind during the course, would it prepare me for the certification? The quick, easy is a resounding No. The course instructor was quite clear on that with the additional advice that for the certification you should really be using the framework on a day to day basis and feel very comfortable and confident in its usage and methodologies.

Summary

Given everything I’ve written above, I’ll summarise everything in two easy bullet points:

New to Zend Framework: This course does exactly what you’d expect, it gives you a nice introduction to the framework and a good grounding on the basics from which you can build. The course seems to generate interest and enthusiasm for the framework amongst developers.
Used the Zend Framework: While it was nice to shore up some of the very basics I felt the time, effort, and funds to take the course could of been better spent elsewhere. It will be nice to seeÂ Zend create a new higher level course to take developers to the next level – at least to the standard of certification and beyond. For that I would sign up immediately.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

[notice] child pid XXXX exit signal Segmentation fault (11)

Steven Lloyd Watkin — Sun, 11 Oct 2009 18:09:52 +0000

If you’ve recently upgraded PHP or Apache you maybe coming up against the issue of your webserver returning blank pages, and throwing error messages into your logs with no idea why, here’s one possible way to fix it…

I’ve had this problem a couple of times recently after upgrading Apache or PHP on a virtual machine. The first time I noticed the error I simply reverted to a backup of my VM but the second time I realised I’d have to look into the issue.

The first time I noticed the issue some of my web pages were being served as blank files while the others worked absolutely fine. After some investigation I noted that apache was writing out to /var/log/http/error_log with the following message repeatidly:

[notice] child pid XXXX exit signal Segmentation fault (11)

There’s not allot to go by on-line, and most of the pages about it trail off to nothing. That said, I narrowed down the issue to PHP crashing when trying to unneeded dynamic libraries.

Looking at my php.ini (/etc/php.ini) I commented out all of the dynamic libraries loaded planning on commenting them back in as required. The two I had to take out where pdo.so and mysql.so.

Once these were removed all my web pages were being served fine, just as before the PHP/Apache update.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Wireless on Acer 5002 WLMi on Linux (Fedora 11)

Steven Lloyd Watkin — Sat, 11 Jul 2009 21:48:05 +0000

As I’ve spent another few hours today without internet access I thought I’d better get this written down so that next time I mess my laptop up the information is easy to fix.

Basically to get wireless drivers working for an Acer 5002 WLMi you’ll need to use b43-fwcutter. Instructions can be found here: Linux Wireless B43.

Easy once the information is located.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Html Table Generating Class (HtmlTable.php)

Steven Lloyd Watkin — Tue, 05 May 2009 21:11:26 +0000

I required to generate a HTML table from PHP data for a project that I was working on. Unfortunately after a bit of Googling I couldn’t really find anything that was suitable so I decided to create my own.

I realise that this may not be the best table generating class ever and there’s probably a few bugs still in it, but I’d prefer to share and if anyone wants a better version I’ll look into it :) I originally wanted to create a table built up of seperate objects e.g. cell object, row object (built up of cell objects), but I realised the overhead on this would be HUGE! So I’ve gone back to an array setup.

The class will calculate the maximum number of columns used in the header, footer, or body and write out that number of columns. Headers, Footers, and the Caption are optional. Attributes are made up of an array where the key is the attribute name and the value is the attribute value. Attributes are applied to table tag (on table creation – class initialisation), table rows (array[row][attribute]), and table cells (array[row][column][attribute]) this applies for headers, footers, and the table body.

Example

I always find the best way to look at something is to use an example so here we go:

$table = new HtmlTable( array('class' =>'sortable',
                'style' => 'width: 550px;'));
$header[0][0]['content'] = 'ID';
$header[0][1]['content'] = 'Title';
$header[0][2]['content'] = 'Date';
$header[0][3]['content'] = 'Site';

$table->addHeader($header);
$i = 0;
foreach ($newsList AS $newsItem) {
$itemLink = "{$newsItem['title']}";
$tableBody[$i] = array(0 => array('content' => $newsItem['id'],
'attributes' => array('style' => 'font-weight: bold;')),
1 => array('content' => $itemLink),
2 => array('content' => $newsItem['date_posted']),
3 => array('content' => $newsItem['site']) );
$i++;
}
$table->addBody($tableBody);
echo $table->drawTable();

ID	Title	Date	Site
9	News 1	2009-04-04 10:40:00	site name
10	News 2	2009-04-04 12:44:11	site name

And the HTML looks like this:

<table class = "sortable" style = "width: 550px;" >
<thead>
	<tr >
		<th >IDth>

		<th >Titleth>
		<th >Dateth>
		<th >Siteth>
tr>
thead>
<tbody>
	<tr >
		<td style = "font-weight: bold;" >9td>

		<td >News 1td>
		<td >2009-04-04 10:40:00td>
		<td >site nametd>
tr>
	<tr >
		<td style = "font-weight: bold;" >10td>
		<td >News 2td>

		<td >2009-04-04 12:40:00td>
		<td >site nametd>
tr>

tbody>
table>

Code

HtmlTable.php
If you have any comments please make them below, it would be great to hear some. If you end up using it also let me know and I’ll extend it in the future :)

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Log to DB using Zend Framework

Steven Lloyd Watkin — Tue, 14 Apr 2009 21:06:30 +0000

I’ve managed to get a site up and running with the Zend Framework, everything is logging nicely to FireBug/FirePHP so next step was to log to the DB. I also wanted to log some additional information using the framework such as user agent, date and time, get and post variables. So to extend the manual a little here’s what I did:

// Set up logging to DB $db = Zend_Registry::get('dbAdapters'); $db = $db['general'];


$columnMapping = array(	'priority' 	=> 'priority',

						'message' 	=> 'message',

						'datetime' 	=> 'timestamp',

						'user_agent'=> 'user_agent',

						'get_vars'	=> 'get_vars',

						'post_vars'	=> 'post_vars',

						'site'		=> 'site'

						);
$writerDb   = new Zend_Log_Writer_Db($db, 'error_logging', $columnMapping);

$logger 	= new Zend_Log($writerDb);
$logger->setEventItem('datetime',date('Y-m-d H:i:s'));

$logger->setEventItem('user_agent',$_SERVER['HTTP_USER_AGENT']);

$logger->setEventItem('get_vars',print_r($_GET,true));

$logger->setEventItem('post_vars',print_r($_POST,true));

$logger->setEventItem('site',SITE);

$logger->info('Informational message');

Where the array keys in $columnMapping are my column names. ‘Priority’ and ‘message’ are understood by Zend_Log_Writers but the additional fields were added to give me some additional information.

Obviously this assumes that you have logging working using one of the other writers first :)

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

UK Names Directory and Facebook Application

Steven Lloyd Watkin — Fri, 10 Apr 2009 09:33:05 +0000

One of the fun things I’ve been involved in whilst working for Tracesmart is the names directory. Basically we took our huge information database and pulled out a big pile of stats about various names.

The process is all one big SEO effort in order to draw more traffic to the site, but it also provides some interesting information and name statistics.

Here’s an example for my name Lloyd Watkin, or I suppose more correctly Steven Watkin. Take a minute to look up your name on the Tracesmart site, might be interesting ;)

We’ve also got names statistics data on the names directory page which tells you what names have been searched and how recently. Its quite interesting to see famous people’s names fly up the stats when they appear in the news. A recent example is that of Liam Neeson who’s wife died from a skiing accident recently, although obviously people aren’t quite right on the spelling (hence the huge numbers of hits on this name) — Liam Nilson.

Facebook Application

Once the names directory was created we set about creating a facebook application which we duely named ‘My Name‘. The facebook application is an extension of the names directory and provides some additional statistics. In order to add the names directory to your facebook account please visit:

http://apps.facebook.com/my-name/

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Flickr and Yahoo!: Forgotten Login Details

Steven Lloyd Watkin — Thu, 02 Apr 2009 19:31:22 +0000

A few months ago I changed my Flickr password so that a friend could upload some shots to it. A couple of months later I find I’ve forgotten that password and trying to remember all my secret details from Yahoo! is just a nightmare. I had my Yahoo! email address for several years and I wasn’t sure when or where (I was living) when I signed up – oh just to make this clearer Flickr uses Yahoo! login system :)

Yahoo! were no help either, they wanted exact details and would reject anything that didn’t match. Plus I wasn’t allowed to provide a list of possible matching details it had to be right on every detail otherwise it would be rejected. I can understand the security but this was getting a pain in the arse, especially as I could retrieve my user name to my usual email address without any trouble (why couldn’t you send a reset there!?!?!). On top of all this they wanted the details sent to their offices in California either by mail or fax, boo!

Step back to Flickr who kindly send an email to my registered email address asking me to detach my Yahoo! account from my Flickr account. Hooray I could then sign up with a new Yahoo! account and reattach my Flickr account to this new login.

After waiting at least a week each time for Yahoo! to reply Flickr got this all sorted within 2-3 hours!

Well done Flickr, thankfully you’re out there looking after your customers :)

Now for everyone’s boredom here’s yet another link to my Flickr account

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Hiking from Ystradgynlais up to Abercraf

Steven Lloyd Watkin — Sat, 14 Mar 2009 17:45:19 +0000

Went hiking with the guys from Pontardawe today. we got a taxi up to Ystradgynlais and started a hike there after a breakfast in Merlyns.Â As a note a few doors down there’s a cafe called Mimosa which is named after the boat which took the first Welsh settlers to Patagonia. The cafe was opened many yeas later by descendants of one of those original families who had returned to Wales.

We headed over to the Neath to Brecon Railway, and then up the hill to see the site of an old powered tram system with powered by something called “the machine” or “the engine”, I forget. Anyway at the top of the hill is the ruins of the old machine house which you can see in my photos. From there we headed north towards the Cribarth (and the pub). The Sarn Helen Open Cast meant that we had to detour and head back down to the railway line.

We headed up the railway line forÂ a bit (its being changed to a cycle path by the open cast people), and cut off to cross the valley to head up the Cribarth. We came across a circular building which was used to feed cattle of a Scottish design (McTurk brothers?).Â There was a dead sheep inside so obviously it wasn’t working so well any more ;)

We followed the Tawe for awhile before headnig up the Cribarth and eventually towards the Ancient Briton for a few beers and watch the Italy vs Wales rugby match.

Heres some photos: Ystradgynlais to Abercraf hike

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Inspired… and guilted!

Steven Lloyd Watkin — Wed, 11 Mar 2009 20:51:31 +0000

I’ve recently set up a blog for one of the guys in work – Matthew Hopkins – and seeing his first few posts has made me think that maybe I should add a few posts as its been a good while and lots has happened over 2008. Another one of the guys in work does himself a blog to, good old Stephen Griffiths so take a look there too.

So I’m going to plod on and add some retrospective posts about what 2008 held for me and how 2009 is shaping up :)

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

2008 WTF?!?

Steven Lloyd Watkin — Thu, 01 Jan 2009 00:00:12 +0000

2008 was a strange old year, there’s several things I really enjoyed about it,Â and there’s some things that have now been forgotten and moved on with….

Diving

I haven’t done much in the way of scuba diving I think I managed around about 10 dives all year. One of those dives was the scuba santas event at Vobster Quay which was more successful than my wildest dreams. In the end we raised over Â£1000 for the RNLI, got coverage on BBC and ITV, and had around 80 divers in the water. Vobster was a sea of scuba diving santas for the day! I really hope I get to do mroe diving in 2009 (lots more!), getting back into it will involve lots of training to get comfortable again. I miss the days of getting on the boat, sun beaming down, grabbing a lobster, packing upÂ Â and heading home for a few beers.

Running

At the end of January 2008 I was involved in a car crash, hit from behind whilst stopped, nothing major or so I thought. Over several months my neck pain got progressively worse stopping me running. In the end I contacted an accident claims company who started getting me treatment with a chiropractor in Cardiff which I’m still attending (March 2009 – the claim still isn’t settled either). Running is another one of those things I want to get back to, I did start with Miles but between illnesses, injuries, and him eventually moving to Cambridge we only got out a handful of times. I really need a good kick up the bum to get going again, although my one consolation is that I have been cycling back and fore to work for months which is doing me some good.

Moved

In July 2008 I moved into a flat with Megan, we’re getting through the painting and decorating and getting it looking really niceÂ – although we’re sort of dragging our heels with the painting….still. The flat is in Adamsdown in Cardiff (I never seem to move far) and is suiting us just great.

Work

Work was all over the place in 2008, I started off drifting a little before taking up work at a letting agent as manager. This helped no end with the estate and letting agency software that I was developing. My plan for the estate and letting agency software is to release it as a low cost product when I have more time to clear it up. From what I’m told about competing products on the market it is easier to use and performs many or more of the same functions in a better way :)

During April 2008 coming up to my 22nd birthday both Megan and my mother started pushing for me to ‘get a career’, which a hard idea to swallow was really what I needed. Running my own company was fun but didn’t provide the stability needed and earning a pittance at the letting agency wasn’t a long term career goal. It was always a case of wait until I get my software finished and I’ll start selling it, but time was always against me.

What followed was several interviews with some great companies in some great areas, full of promise of moving to a new area, fresh start seemed great. The best job offer was with a large hosting provider in the Worcester area and seemed ideal. So what happened? Well I had developed an online SMS solution back around christmas that had been sitting around not doing much which investors were found for. Excellent I thought and off we went. All didn’t end well and after several months of broken promises and rising stress levels I made a stand and walked out. Working 60 hours a week, coming home and doing another 3-4 hours plus working all weekend and for all the stress it was one of the best decisions I’ve ever made. (Here I’d like to thank Megan, my parents, and my friends that stood by me, I was a stress-headed dick at times, and I’m sorry)

Four weeks later I landed myself a job with a company called Tracesmart in Cardiff Bay as a PHP Developer. After passing the tests and the interviews I started with a whole pile of enthusiasm (as you can imagine things had been a bit low for sometime). Tracesmart perform people searching, people tracing, electoral roll data handling, involving huge quantities of data. I’ve been there now for about 5 months (given I’m actually writing this in March 2009) and I love every minute of it. I’m pushing myself every day, have a great working environment (my desk overlooks the bay!), good colleagues! I’ve been involved in some good projects (other posts to follow) and I’m really proud of the work I’m doing there.

2009….

Despite starting 2009 in Germany suffering from the Novo virus 2009 is great so far. Megan and I spentÂ New Years time in Germany and Amsterdam. Amsterdam is beautiful and the Anne Frank house had a real profound effect on me, somewhere we’ll definately be returning to. Work is going really well. My neck is getting better and better so in a couple of weeks time I’ll head out for those first few awful runs once again. The only thing I really need now is a good few dives (or a dive holiday) to get back into the swing of things!

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Scuba Santa’s Website Launched

Steven Lloyd Watkin — Sat, 22 Nov 2008 15:55:39 +0000

Given the growing size of this years scuba santas event to be held at Vobster Quay on the 20th December 2008 I thought I should create a website to collect all the information in one place instead of having it spread between Yorkshire Divers (YD) and facebook.

So here’s the site Scuba Santas – http://www.scuba-santas.co.uk

Links are appreciated as this is a fun event that I organise to raise money for the RNLI each year.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Scuba Santas 2008 – Up and Running…

Steven Lloyd Watkin — Fri, 05 Sep 2008 14:54:28 +0000

Scuba Santas 2008 is now up and running. Planning is underway for the 2008 dives organised through YD for the 20th December 2008.Â The event is being controlled through facebook events.

There’s also a first individual post on YD here about what date to run the event and to get some provisional names for diving santas: Scuba Diving Santas 2008 – on Yorkshire Divers.

The event will once again be held at Vobster Quay, we’ve applied for another Guiness World Record, and hopefully get more visits from news crews such as those we had last year from the BBC.

More news will be posted once we’ve decided on a charity and things are moving along :)

Here’s more about last years event: Scuba Santas 2007.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

IanDennisTechnical.co.uk Launches – Trimix, Advanced Nitrox Training

Steven Lloyd Watkin — Thu, 04 Sep 2008 14:39:12 +0000

Ian Dennis Technical has now been launched. Simply a shift of domains for Advanced Diver Training was a quick inexpensive website set up for a scuba diving instructor. So for all your uk trimix training you know where to go! Ian Dennis Technical – IANTD training in nitrox and trimix

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Disabled Form Fields are not submitted…

Steven Lloyd Watkin — Thu, 04 Sep 2008 11:01:10 +0000

As a quick note, as part of the ongoing estate agency software project I’ve gone back to editing date entry within the system. Now instead of allowing users to enter any old bit of text I’ve switched over to a Javascript date/time picker (for the short term) that allows user to ‘pick’ and date and time from a pop up window.

Now, for my old systemÂ the picker would pop up when the user clicked on the input field. This however would allow the user to ignore the text field and still go and type any old thing into the field. This wasn’t ideal. So my solution was to ‘disable‘ the form field and put a span next to it with an onclick event calling the date/time picker. This meant that the user was almost forced into using the nice date/time picker so dates would go in with the correct format (although that still has to be validated!).

So I started passing my nicely formatted date-time(s) through the form and ~~picking~~ trying to pick them up with PHP‘s $_GET or $_POST variables. Unfortunately’disabled’ fields are not passed through form submissions This was very annoying. The solution was to enable the input element using javascript when the form was submitted using the onsubmit event.

This meant that the form data was passed through as required and was nicely formatted ready for validation…Â

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Search Array Keys and Return Matches

Steven Lloyd Watkin — Wed, 13 Aug 2008 16:26:57 +0000

I’ve had the need to search through an array and return the elements that have keys that match a search term. So I thought I’d share.

‘Why would I need this?’ would probably be the first question you’d ask, well if I wanted to search through an array for all the elements that related to the dimensions of an item then I could pull out the appropriate keys by using this little function:

 $data)
Â Â {
Â Â Â if (strpos(strtolower($key),$search) === 0) { $returnArray[$key] = $data; }Â 
Â Â }
Â Â return $returnArray;
Â } else
Â {Â // User hasn't subimitted an array...
Â Â return false;
Â }
}
?>

So for example if I wanted to search an array of item data for the dimension data then I could do the following (and print to screen presumably):

$dimensions = searchArrayKeys($itemArray,'dimension');

Which would return something like:

print_r($dimensions);
Array
(
    [dimension_height] => 20
    [dimension_width] => 30
    [dimension_depth] => 40
)

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Custom php.ini with Plesk

Steven Lloyd Watkin — Wed, 13 Aug 2008 15:53:13 +0000

We have a dedicated server with ukFastÂ on which we run a number of domains that exist on virtual servers running under Plesk. Anyway, on most of my domains I want to hide away warnings and for security keep errors to the log files, however on some of my domains (such as those I’m developing on) I want to show every little nasty warning and error that appears.

If I edit my PHP.INI”>php.ini this affects each and every domain on the server, not a good idea especially when several of the sites are fully live versions Â So in order to get around this you need to put a file called vhosts.conf in your conf directory, for example, /var/www/vhosts//conf/vhost.conf.

Once you’ve added your additional code, again for example I included,

php_value error_reporting E_ALL
php_value display_errors On

followed by running the following command (this reconfigures all of your domains its just ~~quicker~~ lazier than typing in the domain name ,

# /usr/local/psa/admin/bin/websrvmng -a

If you’re feeling a bit nimble then you can always type out the full version which is,

# /usr/local/psa/admin/sbin/websrvmng -u --vhost-name=

After this command if you look to httpd.include file, will see that your httpd.include will have an include line for your vhost.conf, something like this:

Include /srv/www/vhosts/domain.com/conf/vhost.conf

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Diving Rheola Lakes, Glynneath

Steven Lloyd Watkin — Sat, 02 Aug 2008 09:00:38 +0000

Hearing about a new dive site through YD, it being cheaper (Â£5) and closer (well for Simon) than the NDAC we decided to head down to check it out. The site is north of Neath on the A465 and is an easy drive from the M4. The site is run by Steve of Quest Marine (site details on link).

There are two lakes at the site, one for powerboating and one forÂ quieter activities (such asÂ diving!). Â Parking is right up against the water and for the entire day we were the only two divers there . The smaller lake isn’t huge but an ok size for a smaller number of divers. There’s toilets at the site but not much else, although there’s a burger van a short drive back to the entrance to the site and a place you can buy junk food just around the corner.

The lakes were created as they were dug out for rock to build the heads of the valleys road and have naturally filled up over time, due to that there’s no fish life although plenty of plants. Adding some fish would be quite nice and would draw in some additional money from fishermen, although there were some kids fishing the bigger lake so I guess that there’s probably some in there.

The water was very clear with a good 5-6m visibility, although if you knocked theÂ bottom lots of suspended matter was sent upwards and took a while to settle. The suspended matter was much preferrable to theÂ black dust knocked up if you find a patch without plants. I did this to test what it was like butÂ forgot to take some video to demonstrate

Â Â

We were told that the max depth was 12m with the lake sloping down in 2m steps but we reached a max of 10.8metres. After about 20 minutes (+ 1 golf ball) we both got a bit bored of the site but pushed the dive to 40 minutes. The second dive we went in with a plan and did several run times with stops at 9m, 6m, and 3m both up and down.

In summary, a good cheap site at Â£5, not much depth but good for practice.Â I wouldn’t go for a pleasure dive, there needs to be a purpose, maybe if the site develops a few objects would go in but I don’t know what the owners plans are there!

Diving Rheola Lakes, Glynneath – 2nd August 2008

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

PHP Variables not existing and forms

Steven Lloyd Watkin — Thu, 31 Jul 2008 10:36:59 +0000

I came across something I wasn’t aware of today that is very handy…. the error control operator ‘@’Â in PHP.

So for some background to how I came across this problem, in my web applications I tend to use the same form to insert data into a mySQL database. If I have an some sort of ‘id’ number set via a GET variable then that triggers a load from the database, otherwise I’m adding new data.

In my form elements I often set the value of the form field to something like this works fine when loading data but when I’m inserting new data I get the form filled in as follows (note on my login or when I’m testing I have error reporting set to E_ALL so I can make sure everything is running correctly):

Notice: Undefined variable: data in /home/public_html/dataEdit.php on line 69

So what I eventually ended up doing was writing longer (uglier) code which said, . Not exactly great

Then I stumbled across the error control operator in PHP, by simply placing an ‘@’ symbol before an operator (basically anything that can return a result) the above problem is solved. For example I can now make my for value parameter <?=@$data['fieldValue'];?>. If the variable is not set (i.e. an error would be generated) then the error is suppressed.

Note: Using the ‘@’ symbol can supress major errors and hide the fact that part of your site isn’t running correctly, so be careful!

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Moving Server, Safe Mode, and File Ownership…

Steven Lloyd Watkin — Tue, 29 Jul 2008 15:42:58 +0000

I recently moved some of the web applications that I’ve written over to a new dedicated server with UKFast. The new server I had with my previous terrible web host, was setup in a much less secure way (PHP Safe Mode”>safe_mode off for example) than the new one so I was finding that several scripts weren’t working.

One of the major annoyances was php file uploads used to put property pictures onto the site. With error reporting switched to maximum and Display Errors switched on in my php.ini file I was able to see the full list of errors. One of the major complaints stated something along the lines of…

PHP Warning:Â Unknown(): SAFE MODE Restriction in effect.Â The script whose uid is 1 is not allowed to access /path/to/script.php owned by uid 26658 in Unknown on line 0;

Many of the directories and image files were created by the old site and so file ownership was a major problem. Technical support suggested turning off safe mode as a short term fix and then working around it in the longer term. I wasn’t quite happy with this solution so spent some time thinking through the problem.

I knew that you could change the ownership of a file by logging in as root via SSH and running the following commands…

<>chown user:group filename (also see chgrp)
(You can use the -R switch to chown all the files in your site)

I started by setting the file and group ownership of the files on my site to 26658 (if we’re using the above example!). This seemed to sort the problem until that is I tried to FTP files to and from the site, resulting in a permission denied and/or file not found errors.

So after playing around with several uid’s I had a cup of tea and thought through the problems. For me to upload files via FTP then the owner of the file is going to have to match the FTP username. So I set the owner of the file as the username I used to log in via FTP. Secondly, the webserver is going to want to access the files so I set the group ownership as ‘apache‘.

This didn’t quite work, but setting the /tmp directory ownership to nobody (chown nobody /tmp) seems to have fixed the problems :)

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

FREE Board Management Software for Estates and Letting Agents

Steven Lloyd Watkin — Thu, 19 Jun 2008 11:23:09 +0000

As a result of feedback from our agents I’ve now added FREE board management software to the property SMS software that I’ve written.

I’ve created the system so you can make use of the board management or not. If agents use the software then the website will send out an email to their board agent at the end of the day to instruct them on board placement (there’s more detail in there than that :)).

Anyway please feel free to take a look at the property SMS system that I’ve written.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Instant Property Information (IPI)

Steven Lloyd Watkin — Sat, 14 Jun 2008 13:46:54 +0000

Â Property Information via SMS

Evilprofessor Designs have created a fully bespoke SMS application for a new company called Instant Property Information. IPI provide a service whereby instant information can be delivered to people interested in property 24 hours a day 7 days a week even when the office is closed. The system also includes a FREE board management system, and can send out scheduled text messages for estates and letting agents. Unlike most other systems out there we’ve done the maths and have made sure that this system is FREE for agents.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Instant Property Information via SMS

Steven Lloyd Watkin — Wed, 11 Jun 2008 14:00:46 +0000

I’ve just taken up a position with a new company providing property information by SMS / Text Messages. Best of all we worked out a way that we can offer this service to estate and letting agents for FREE! No ties! No contract!

How does instant property information work? Basically, a prospective tenant or house buyer Â sees a property that they like and texts a code on the property board. Instantly they recieve property information back from your company with your contact details. Ten minutes later you receive an email containing their contact details and the property that they were insterested in, so you can call them giving them additional details and to arrange viewings.

This was developed with a letting agent in Cardiff so we’ve got plenty of industry knowledge to answer some of the questions that other systems out there do not address.

Hopefully you’ll be seeing this service come to a your streetÂ in the near future, instant property information.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Santa Dives – Vobster Quay 2007

Steven Lloyd Watkin — Sat, 22 Dec 2007 10:30:08 +0000

Packing up the car on aÂ cold frostyÂ morning I wondered “whatÂ am I thinking“. Actually… that’s much like most early start dive trips except this morning I was shivering a little :( I was heading down to Vobster Quay in Somerset to meet several other equally foolish people for an interesting new eventÂ organised between members of Yorkshire DiversÂ (YD).

The drive down from Cardiff was interesting due to the amount of fog and I had to crawl the car through Bristol. Upon reaching the dive site we saw that the water was in the same state as many people from around the country had been the previous night…. steaming ;)

So why had a small hardy bunch gathered in the middle of Somerset on a freezing December morning? Well its the age old story of getting dressed up as father christmas and jumping into cold water to raise money for charity. Every year members of YD work hard and selflessly to raise money for various charities and this year it was the turn of the RNLI, appreciated by many divers in the UK.

During the dive the Santas were joined byÂ a christmas fairy, a snowman, and a pagan viking (celebrating the Winter Solstice). It doesn’t exactly sound like the most seraphic underwater scene but it was the good kind of strange being surrounded by such a group of people underwater. One of the nice things is thatÂ the group wasÂ diving everything from single tanks to twinsets and rebreathers. We all came from different diving backgrounds PADI, BSAC, TDI, IANTD, GUE but we were all there to have a bit of winter diving fun asÂ one big underwater group :)

A chilly first dive of around 45 minutes took us over to the rear end of an aeroplane fusilage where everyone posted for festive underwater christmas photos. Floating inside an aeroplane fusilage, underwater, hearing two guys singing “We’re walking in the air” has to count as one of the most surreal experiences of my life! Maybe narcosis does take significant effect at less than 30 metres as I spent most of the dive smiling or laughing mask filling with water all the time :)

We finished on a boat wreck which had been festively decorated by staff at Vobster Quay. Those who turned up to keep watch on the surface later told the group how people had exited the water looking confused asking “Umm did anyone else see all the father christmas’ underwater?”.

On surfacing all involved took readily to a nice hot cup of tea, some some mince pies, and pulling christmas crackers. Those celebrating the winter solstice were soon offering people a small cup of meade to warm the soul after a cold dive, another welcome addition to the day.

LaterÂ we were joined by an enthusitastic young journalist who wanted to come along for a dive to experienceÂ the group of Santas. “Crazy“Â I thought, but we had a spare Santa outfit, soÂ as soon as we could feel our extremities then once again it was time to jump in. We obtained some interesting video footage for the BBC which included several santas disappearing into in long dark tunnel and some very festive rebreather divers.

Upon exiting the water maybe a little of the enthusiasm was lost but there were definate smiles from our new friend. Everybody should dive with a group of Santas!

Overall the whole group had a fun enjoyable day despite the temperatures, and we certainly managed to spread a little of that Christmas magic for everyone at the site that day. I’d like to thank Tim from Vobster Quay for the use of their facilities and donating Â£5 for every scuba diving Santa, Gareth Lock from “Images of Life” for his excellent underwater photography skills, everyone who took part today, and also everyone that has donated money for the event. Donations can be collected until the mid-Feburary 2008 via our JustGiving page: http://www.justgiving.com/ScubaSantas/.

See you next year!

Lloyd Watkin

See more pictures here: Scuba Santas 2007

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Merthyr Mawr Christmas Pudding Challenge 10k 2007

Steven Lloyd Watkin — Sun, 16 Dec 2007 11:00:05 +0000

I’d been looking forward to this race for a good couple of months! The Christmas Pudding challenge promised a big sand dune and a spell of stream running. I was not disappointed. My first race at Merthyr Mawr, before this race I didn’t even know that the highest sand dune in Europe was almost in my back yard. After parking the car a couple of miles away and running up to the start I saw the monster in front of me *gulp* much bigger than I was expecting.

Â Here’s a picture of the sand dune from near the top as you can see its a big one at about 300ft! The way up was tough and with lots of people I couldn’t go any faster even if I’d wanted to. Anyway there was a bit of congestion at the top of the dune so there was a few minutes of waiting around (good time for everyone to catch their breath). The next thing I know I’m plummeting straight down the other side at full speed, half worried I’m going to bowl over at any moment, but my its good fun ;)

From there we head out across a flat wind swept area before what I heard to refer as the ‘orrible hill, and it was just that. A long steep hill up to the golf course (at least it wasn’t covered in sand too). So did the usual trick, drop the arms, reduce the stride length and switch into “Disney’s LittleÂ Engine that Could” mode….

“I know I can, I’m sure I can, I know I can, I’m sure I can”

…which slowly became, “I’m not sure I can, I know my legs hurt” :( Slowly but surely I got to the top of the hill and had some nice views down to the sea.

I figured the mid-point couldn’t be far away at thisÂ time and upon approaching not only were we offered water but mulled wine and mince pies. Now being the highly professional athlete I am of course I opted for the mulled wine, vitamins, water, warmth it all made sense…..surely? A good cup of hot mulled wine it was and with the long slow decent I started flying along – alcohol fueled possibly ;)

Around back to the dunes which again made the running a little tough and in the back of my mind the stream was coming. Next thing I know we’re running through a shallowÂ bit of water,Â some tried to skirt around the outside but for a laugh I headed through the middle :) Got out of that one when a marshall mentioned there was another couple to go. Lets say the third one was pretty deep and cold, almost got up to places I don’t wish to have frozen off, but fortunately the crowd was spared me shreaking like a girl had that happened!

Post race I collected my soup and bread roll and Christmas Pudding and ran back to a warm car for the drive home accompanied by some loud JayZ courtesy of Radio 1.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Cardiff Santa Dash 2007

Steven Lloyd Watkin — Sun, 09 Dec 2007 12:30:11 +0000

Â This was to be a quick 5km race around Bute Park in Cardiff, however the weather over the past week meant that a last minute change of course was required reducing the course length to around 3km (best guess!). The announcement system was terrible (being polite) so I don’t think many people actually had much of an idea what was going on.

The race went ok, nothing much to report except although things in the gym has been going well it doesn’t make up for a lack of running in the outdoors! I have the Merthyr Mawr Christmas pudding next weekend which I’m really looking forward to although the weather has prevented much running outside. This will need to fixed over the next week :)

Cardiff Santa Dash 2007 Photos

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Advanced Diver Training

Steven Lloyd Watkin — Sat, 01 Dec 2007 12:00:46 +0000

Advanced Diver Training was a quick inexpensive website set up for a scuba diving instructor. The budget was small but this does not mean that the quality of the end product had to be low. The site was created, edited, and up and running within 3 days at a small cost. We hope you agree that the site has a professional look and the site owner is able to update and edit the site without further intevention of a website designer. This website was developed with the open source software wordpress.

Advanced Diver Training – PADI/IANTD training in nitrox and trimix

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Keyfinder

Steven Lloyd Watkin — Thu, 01 Nov 2007 12:00:43 +0000

Keyfinder required a new online store for their business. Open source software solutions wouldn’t have provided the features that they required, therefore Evilprofessor Designs stepped in and created them a fully custom webstore with full PayPal IPN integration. Keyfinder staff can fully customise the website through an admin interface which includes updating orders and registations, editing page text, and add/edit/update product listings.

Keyfinder – Security and peace of mind for your keys…

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Tour of Torfaen Half Marathon

Steven Lloyd Watkin — Sun, 28 Oct 2007 10:00:20 +0000

What am I doing? I’ve been in bed all week with a bad cold, I’ve managed one run since the Cardiff Half where I pulled my left calf, and now I’m stood out in the freezing driving rain about to run another one!!! Standing at the start line, soaked to the bones already, the hooter goes…. here we go again :)

As I said, I’d been in bed with a cold the previous week before the race, was hit and miss about whether I’d do the race but Friday morning I was feeling a bit better (besides the horrendus cough) so I thought why not. I knew I’d only be sitting around on Sunday morning wishing I’d done the race anyway.

Waking up Sunday morning I looked out of the window and oh bugger was it raining hard, the warm bed looked awfully inviting, but no I was going. Arriving at Cwmbran Stadium if anything the weather had got a little worse, but I climbed onto the coach for the trip up to Blaenavon for the start. Runners gathered in the rugby club before the race, and despite the conditions outside everyone was in high spirits and lots of laughs going around :)

I decided that I was going to try and cover the first part of the race as fast as I could, I knew without much training I wouldn’t really have the stamina to keep going for the entire race. So I belted it at the start planning on just surviving the last part of the course. I managed to cover the first 5km in just over 20 minutes (good news for the next 5km race!).

All the way around everyone out supporting (or just walking the dog), the PCSOs directing traffic, and all the other runnersÂ were very friendly and encouraging. That’s the main difference that struck me about this compared to the Cardiff Half.

As predicted after about 9 miles I could feel my legs getting heavy and my pace slowed right down. The wind had picked up again and at times was holding runners on the spot, not nice at this point in the race. By 11 miles my chest got tight and I had to stop a couple of times for some coughing fits :( Still arriving in the stadium for a lap and a half I managed to grab a little more energy from somewhere.

I eventually finished in 1:56:33, drenched, tired, aching, but with a big smile on my face! More like a time from a training run, but that’s fine given the circumstances. I’ll definately be back next year, hopefully I’ll be in much better physical state to run and the weather gods will be smiling on the event ;)

All that said it was nice to get home to a hot bath and some cold beers :)

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Stuck on 4096 characters???

Steven Lloyd Watkin — Fri, 26 Oct 2007 14:21:58 +0000

I’ve come up with a problem recently returning data via an XML feed using AJAX. In my case I was returning a list of images each having several links to perform various tasks (e.g. set as main image, add as thumbnail, add full image, etc).

The first load would go through fine and all images and their links would appear as I’d expect. There was also no problems in IE7, Safari (on Windows), or Opera 9+. When it came to Firefox 2 the image list would be cut short! Editing the javascript it basically seems that the returned data would be cut off at 4096 characters.

It took a while to track down the result. I thought initially this might of been a javascript maximum string length problem, but that other browsers just let the issue slip. Not that either. Anyway it turns out that text nodes are split at 4096 characters.

It’s quite simple to fix you simply need to normalise() your returned data in your AJAX script. Simply add the second line once you’ve gathered your XML data:

xmlResponse = xmlHttp.responseXML; xmlResponse.normalize();

This will solve the 4096 character limit when using Firefox 2 or any other browser I haven’t checked that doesn’t normalise the data automatically.

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Quick and easy $_POST security…

Steven Lloyd Watkin — Thu, 25 Oct 2007 19:44:50 +0000

A quick and easy way to protect yourself from mySQL injection attacks in PHP is to use…

$sql = "insert into table set "; Â foreach ($_POST as $key => $data) Â { Â Â Â Â $sql .= $key." = '".strip_tags(htmlentities(addslashes($data)))."',"; Â } Â $sql .= mysql_query(rtrim($sql,',').";") or die(mysql_error());
What this script does is to take your $_POST data and remove anything malicious from it. Looping over the $_POST data we build up an SQL statement. At the end then we simply execute using with mysql_query.

NOTE: This does not validate your data, it just helps prevent malicious attacks.

Another version of this that I use to to grab all my form information into variables to to place the following code within the foreach loop instead…

eval("$".$key." = '".strip_tags(htmlentities(addslashes($data)))."';");

This makes up a list of variables following your forms field names and strips malicious code from them. This method then allows you to do some form validation before inserting data into your tables ;)

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Cardiff Half Marathon 2007

Steven Lloyd Watkin — Sun, 14 Oct 2007 09:00:09 +0000

So today was it, my first half marathon. Training had gone fairly well, I’d managed to do my over the mountain half training several times my best time on that being 1 hour 57 mins roughly. A few days of tapering, which generally meant I felt rubbish and there I was standing on the start line at 8:30am. I finished in 1 hour 48 mins 44 seconds.

I wasn’t as nervous for this race as I was for the Great Wales Run, maybe it’s because I was better prepared, not as well prepared, or just plain stupid! Either way it was good for my stomach.

The race got going at 9am and started off just fine. I planned to run 8:20 minute miles for the whole race. The good thing about 8:20 minute miles is that it makes the maths easy for getting your pacing right. Every 3 miles is 25 minutes! Getting to the one mile marker I checked my watch that readÂ 4 minutes 20Â seconds, hmm bugger, something had gone wrong thereÂ :(Â It simply meant I had to do a little more maths to work out my pacing but it kept my head busy :)

As I’ve found in other races I had to spent a good part of the start of the race getting around people who’d started way too far up the field. A little annoying and knackers you out a bit from the quick bursts of speed to get around people.

The race itself was rather nice, good to be across the barage again, although I realised after a bit that I was running on the windward side and acting as a windbreak for so many other people, that said the breeze was nice :) Support around the bay was good it’s always nice to have people giving a shout. I kept popping a glucose tablet every 3 miles and having a few mouthfuls of water at the water stations.

Heading back up into the town there was lots of supporters outside the stadium and I almost cried when we passed the burger van at the market. Still only another couple of hours and I’d be sitting in a pub enjoying the fruits of my labour. I was hitting my pacing rather well as far as I could tell which I was quite happy about, meaning that I should get in for approx 1 hour 50 minutes, a time I’d be very happy with!

The last mile was fairly crappy due to the stone work on the ground, I tried to run as much as possible on the grass to avoid that problem. Entering the castle grounds for the finishing straight was awesome there were so many people shouting. It gave me a real lift and I picked up speed and managed to sprint across the line, about to puke as I stopped ;)

Nabbed myself a couple of free bottles of lucozade and sat on the bank waiting for Rhianne to run it in. Fair play she managed to finish in 2 hours 31 minutes much better than she had hoped, well done Rhianne! After a long painful walk home (being overtaken by old people I should add) a quick shower and I was in London Pride heaven. All you can eat Tapas at La Tasca and a few more pints of Brains and all was well with the world again…. now just two weeks until the next one :)

More photos… Cardiff Half Marathon 2007 photos

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Divesigns.com

Steven Lloyd Watkin — Mon, 01 Oct 2007 12:00:26 +0000

Dive Signs required a low cost solution due to being a start up company. Dive Signs are a family run business based in Essex they sell highly reflective vinyl stickers for the scuba diving industry. The solution for Dive Signs was a customised install of Zen Cart, the entire site took less than one week to get up and running and they are already expanding their online catalogue.

Divesigns.com – Highly reflective vinyl signs for scuba diving

Tweet This Post Plurk This Post Buzz This Post Delicious Digg This Post Facebook MySpace Ping This Post Reddit Stumble This Post

Including an RSS Feed on your site…

Steven Lloyd Watkin — Wed, 26 Sep 2007 15:31:00 +0000

I’ve used my RSS feed generator on a few sites now, so I figure it’s about time I share. The final output validates and conforms to RSS2.0 standards so it will import into most things. I’ve even got mine linked to my facebook profile. The full code is at the bottom of the page.

Basically I use my RSS script to generate news feeds for the sites that I write. Examples can be seen on these sites,

But generally you can use the feed generator to serve any information you wish. For example would be the results of a search query where the query variables can be picked up from $_GET[]‘s $_POST[]‘s.

Anyway a question a friend asked me was “What’s the Point in RSS” good question! Click on the link to find out :)

Right anyway onto the script. Firstly we need to tell the browser or news-aggregator that it’s looking at a rss:xml feed,

// Write the RSS header
header("Content-Type: text/xml;charset=utf-8");

It’s at this point that I usually set up my database connection and link to the information I require so usually it’ll be something like,

include_once( "./connect.php" );
$sql = mysql_query("select * from news order by 'datePosted' desc LIMIT 50;" );

I limit to 50 as no one wants to see 1000′s of posts going back years and years unless they specifically ask for it :) Then we start writing out the ‘introduction’ part of the feed,

echo "";
echo "";
echo "";
echo "";
echo "YOUR SITE/FEED DESCRIPTION