My article about the queues

development, php, php classes Comments Off on My article about the queues

I wrote an article how queues can speed up your application and make your users happier.

Queues are everywhere. When you delete a photo on your iPhone, it goes to the Bin and will be deleted from there after 30 days. That is a queue. Without it you will not be able to recover a picture removed by mistake.

Read the full article here.

Specifying CSS class of a Zend_Navigation li element

delete your code, development, php, zend Comments Off on Specifying CSS class of a Zend_Navigation li element

Zend Navigation is a really nice tool to handle menus.

You just specify an array of your menu items:

$pages = array(
    array(
        'label'      => 'Privacy Statement',
        'controller' => 'terms',
        'action'     => 'privacy-statement',
        'class'      => 'firstNav',
    ),
    array(
        'label'      => 'General Terms of Use',
        'controller' => 'terms',
        'action'     => 'general-terms-of-use',
    ),
);

and initialize the navigation object with it — and it works:

$container = new Zend_Navigation($pages);
$this->view->navigation($container)->menu()
    ->setUlClass('nav')
    ->setActiveClass('active');

Result is the following:

<ul class="nav">
    <li class="active">
        <a class="firstNav" href="/terms/privacy-statement">Privacy Statement</a>
    </li>
    <li>
        <a href="/terms/general-terms-of-use">General Terms of Use</a>
    </li>
</ul>

There are options to specify the CSS class of the whole UL tag. If you specify CSS of a menu item (“firstNav” in my example), it’s added to to the A tag, not the LI tag as required by sliced design I have.

Googling shows that people are trying to work-around that by jQuery fixes.

It seems there is a proper way to solve this; you just need to add this option:

$this->view->navigation()->menu()->addPageClassToLi(true);

Enjoy!

Using non-alphanumeric characters in Sitemap URLs

development, encoding, google, links, php Comments Off on Using non-alphanumeric characters in Sitemap URLs

This article in Google Help explains how to deal with special characters in Sitemaps that you can submit to Webmaster tools in order to increase the number of indexed pages of your website.

The main point is: the URLs must contain ASCII symbols only.

It can be done this way:

  • (obvious) ampersand, both quotes and <> symbols must be encoded,
  • Unicode symbols must be encoded, eg. ü must be converted to %FC sequence,
  • URLs that you submit must follow the  RFC-3986

If you use PHP, pay attention to one thing: it seems rawurlencode should be used instead of the usual urlencode since it’s follows the RFC-3986 as stated in PHP documentation.

Problems with download in InternetExplorer

development, php Comments Off on Problems with download in InternetExplorer

Don’t forget to include this simple header, otherwise Internet Explorer users over SSL will get a weird error message that the download cannot be done:

header('Pragma: public');

It’s better to make a filename nice and simple, avoid slashes in it and don’t quote it — again in respect to IE users.

The best way to go is just copy the list of headers to implement the download of a fil from PHP documentation page.

Delete Your Code #7: Right encoding

db, delete your code, development, encoding, mysql, php, server Comments Off on Delete Your Code #7: Right encoding

You might be surprised, but a right choise of the project text encoding can affect the project file size and amount of bugs.

To avoid bugs of wrong presentation of text on your page, make sure that all database entities and the application server (PHP) use the same encoding. That helps to forget about issues connected with text presentation.

On database side, make sure you set the correct encoding to:

  • database,
  • tables,
  • columns,
  • import-export tools parameters (a big source of wrong encoding bugs),
  • corresponding SQL server variables

On application server it’s usually just one query –

SET NAMES utf8

Pay attention, that utf8 might be not the best choise for your project: every non-English character needs 2-6 bytes of memory, so if you built a one-language (local) project with lots of database data, consider using a 1 byte encoding like windows-1251 and save about half of the space on server file system.

Delete Your Code #2: less OOP visibility keywords

delete your code, development, ideas, php Comments Off on Delete Your Code #2: less OOP visibility keywords

Hey! I gonna describe tricks that I use to have less code. Why it is important to have less code, you ask? Less code means less bugs, less support, less developer brains waste.

Today’s trick is extremely simple — when you have a long set of public, protected or private class properties, remove the visibility keyword set to each declaration but define it once as comma separated declaration.

Same applies to class constants as well.

Example:

// Before:
class Cat {
    const KINGDOM = 'Animalia';
    const PHYLUM  = 'Chordata';
    const FAMILY  = 'Felidae';

    public $tail;
    public $whisker = '\/';
    public $head;
    public $legs = array(1,2,3,4);
}

// After:
class Cat {
    const
        KINGDOM = 'Animalia',
        PHYLUM  = 'Chordata',
        FAMILY  = 'Felidae';

    public
        $tail,
        $whisker  = '\/',
        $head,
        $legs = array(1,2,3,4);
}

One benefit is that the code looks clear and it’s much easier to scan rather then to read.

Second benefit is when you need to change the visibility for a property, you don’t have to edit the visibility keyword near the property name (which might be an error prone process when you are tired) — you just move the line up or down, which usually has a shortcut in IDE.

Known disadvantage is that most documenting engines don’t support this ferature.

Sphinx on Windows and error 1067

development, php, server 1 Comment »

I attempted to setup Sphinx on Windows, but was not trivial — problem was that I couldn’t run the SearchD as a Windows service since I got 1067 error (“Process Terminated unexpectedly“).

It’s not obvious but helpful to know how you can get the error message the service crashes with. To do so, go to Control panel → Administration → Event Viewer. Here you can get all notices and error messages that the service produces:

If you get your service running and search tool gives you the correct results, but your PHP application gets FALSE as query result, try to see what SphinxClient’s getLastError() method returns:

$s = new SphinxClient;
$s->setServer("localhost", 3312);
...
$result = $s->query('test');
echo $s->getLastError();

In my case there was a following message:

searchd error: 
  client version is higher than daemon version 
  (client is v.1.22, daemon is v.1.19)

Zend View Helpers inheritance

development, php, plugin, zend Comments Off on Zend View Helpers inheritance

It’s handy to use View Helpers in Zend FW driven project. For example, you want to make an in-place tracker (e.g. Google Analytics) — you create a helper class My_View_Helper_Tracker inherited from Zend_View_Helper, Zend finds its automatically and then you are free to use your helper method:

echo $this->tracker( $trackerID );

The question is what you gonna do if you want a base class for a family of trackers?

It’s not so obvoius due to naming conventions.

Let’s say you want 2 kinds of trackers: Google Analytics and Euroads.

1. You create such files structure:


library
 - My
   - View 
    - Helper
      - Tracker
        Abstract.php
        - Google
          Page.php
        - Euroads
          Owner.php
          Guest.php

2. You name your classes as:

  • My_View_Helper_Tracker_Euroads_Guest (extends My_View_Helper_Tracker_Abstract)
  • … so on …

3. Class methods are:

public function tracker_euroads_guest(...) ...

4. And the trickiest part: the helper call:

echo $this->tracker_Euroads_Guest

Cannot upload a big file?

php, server Comments Off on Cannot upload a big file?

If you want to work with big files uploads (photos, videos, CSV data dumps) you might need to prepare your PHP server for this — increase the values of these variables in your php.ini:

upload_max_filesize = 512M
post_max_size = 512M
memory_limit = 512M

The same should be done if you try to upload SQL dumps by phpMyAdmin and get a correspondent error message.

Currency exchange in your application

db, development, ideas, php, zend 1 Comment »

That’s easy, you need 2 things:

  1. Fresh currencies exchange rates
  2. Some way to excange amount from one currency to another.

This how I did it: get values from European Central Bank (ECB) for step #1 and wrote MySQL user defined function for step #2.

Here is how to export currencies rates from ECB (EUR is a base currency, and I add self rate as 1:1). First I create such database table:

CREATE TABLE IF NOT EXISTS `currency` (
  `code` char(3) NOT NULL DEFAULT '',
  `rate` decimal(10,5) NOT NULL COMMENT 'Rate to EUR got from www.ecb.int',
  PRIMARY KEY (`code`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='Currency rates (regularly updated)';

Now let’s fill it with rates:

<?php

class CurrencyController extends Controller_Ajax_Action {

  public function importAction() {
  
    $db = Zend_Registry::get('db');
    $db->beginTransaction();
    
    $url = 'http://www.ecb.int/stats/eurofxref/eurofxref.zip?1c7a343768baab4322620e3498553b5a';
    try {
      $contents = file_get_contents($url);
      $contents = archive::unzip($contents);
      $contents = explode("\n", $contents);
      
      $names = explode(',', $contents[0]);
      $rates = explode(',', $contents[1]);
      
      $names[] = 'EUR';
      $rates[] = 1;
    
      for ($i = 1; $i < sizeof($names); $i++) {
        if (!(float) $rates[$i]) continue;
        $db->query( sprintf('INSERT INTO `currency`(`code`, `rate`)
              VALUES ("%s", %10.5f)
              ON DUPLICATE KEY UPDATE `rate`=VALUES(`rate`)', 
             trim( $names[$i] ), 
             trim( $rates[$i] )
        ) );
      }
      
      $db->commit();
    } catch ( Exception $O_o ) {
      error_log( $O_o->getMessage() );
      $db->rollback();
    }
    
  }
}

Now let’s create a SQL function for handy converts. Create a udf.sql file and add this in it:

DELIMITER //

DROP  FUNCTION IF EXISTS EXCHANGE;
CREATE FUNCTION EXCHANGE( amount DOUBLE, cFrom CHAR(3), cTo CHAR(3) ) RETURNS DOUBLE READS SQL DATA DETERMINISTIC
    COMMENT 'converts money amount from one currency to another'
BEGIN
    DECLARE rateFrom DOUBLE DEFAULT 0;
    DECLARE rateTo DOUBLE DEFAULT 0;
    
    
    SELECT `rate` INTO rateFrom FROM `currency` WHERE `code` = cFrom;
    SELECT `rate` INTO rateTo   FROM `currency` WHERE `code` = cTo;
    
    IF ISNULL( rateFrom ) OR ISNULL( rateTo ) THEN
        RETURN NULL;
    END IF;
    
    RETURN amount * rateTo / rateFrom;
END; //

DELIMITER ;

and run this command in your shell:

mysql --user=USER --password=PASS DATABASE < udf.sql

This is how you can use this function — how to convert 10 US dollars to Canadian dollars:

SELECT EXCHANGE( 10, 'USD', 'CAD')

which results in $10 = 10.93 Canadian dollars.

P.S. Consider adding the currency export action call to your cron scripts.

P.P.S. A function to unzip the data file can be got at php.net

Ajax controller in Zend project

ideas, php, zend 1 Comment »

I want to share a couple of features I use to handle AJAX requests in projects based on Zend Framework.

1. AJAX request handling

What: some parts of your application can be not loaded if currect request is AJAX.

Why: you don’t need views, templates, some routes — so you can add an AJAX check in your Initializer or bootstrap file and avoid loading not necessary things.

How: Zend Request object has a isXmlHttpRequest method to find out whether it’s AJAX request or not. It’s based on ‘X-Requested-With‘ header, which is sent by jQuery, Prototype, Scriptaculous, YUI and MochiKit frameworks.

2. AJAX Controller

Most AJAX controller’s methods I saw had an exit() inside to not to output Zend’s template — it is a work-around. The proper way to do so is to tell to Zend not to load anything. One step forward is to create an abstract Controller class and inherit all you AJAX classes from it:

/library/Koodix/Controller/Ajax/Action.php:

<?php

require_once 'Zend/Controller/Action.php';

abstract class Koodix_Controller_Ajax_Action 
  extends Zend_Controller_Action 
{
    public function init() {
        //disable the standard layout output
        $this->_helper->layout()->disableLayout();  
        $this->_helper->viewRenderer->setNoRender();
    }

    public function postDispatch() {
        //envelope and output json field
            if( !empty( $this->json ) ) {
                echo json_encode( $this->json );
            }
        }
}

/application/modules/default/controllers/AjaxController.php:

<?php

class AjaxController extends Koodix_Controller_Ajax_Action 
{
 // bla-bla-bla

Take a look at postDispatch method — idea behind it is to convert to JSON and output anything that is set to json field of your controller. If you want to send JSON data in special header (and not in body, like it’s done in my example), you can do it in this method.

Zend Auth Session Expiration Time

development, php, zend Comments Off on Zend Auth Session Expiration Time

After authentication in my project it takes about 10-20 minutes for the auth session to expire, which is not handy — you go to get a snack and see a login screen coming back.

This is how to make the TTL of your auth session longer:

$s = new Zend_Session_Namespace( 
    $this->_auth->getStorage()->getNamespace() );

$s->setExpirationSeconds( strtotime('30 day', 0) );

Now your users will log in for 30 days.

You can call this code in your login action after successful authentication and before the redirect to the landing page.

Sharing work between agents

db, development, ideas, php Comments Off on Sharing work between agents

There are situations when you need to separate processing of big amount of data between several “agents”, e.g.:

  • you have a long list of websites which must be checked for being alive (404 error check) by your web-clawlers;
  • a queue of photos to be resized or videos to be converted;
  • articles that your editors must review;
  • catalogue of blog feeds that your system must import posts from;
  • etc.

The idea to do this is simple:

  1. Give a small piece of big work to an agent.
  2. Mark this piece as given to him (so that none other starts to do the same job) and remember the time stamp when the job was given or when the job becomes obsolete (this agent is dead, let’s give this job to someone else).
  3. If work is done — go to step #1.
  4. After some period of time (1 hour) check all the time stamps, and if some agents didn’t cope with the job, mark the jobs as free so that others could start to work on it.

The problem is between steps #1 and #2 — while you gave a job to Agent 1 and going to mark it as given to him, what if Agent 2 is given by the same job? If you have many Agents, this  can happen at real. This situations is called concurrent read/write.

To overcome this a lock can be used.

In this article I wil explain, how to use locks in Zend project with MySQL database.

First of all, MySQL documentation tells that SELECT .. FOR UPDATE can be used for that purpose. First step is to select records by that statement, and second step is to mark them as locked. Requirements are to use InnoDB storage and to frame these two statements in a transaction.

Happily, Zend_Db_Table_Select has a special method forUpdate() that implements SELECT .. FOR UPDATE statement. Zend_Db can cope with transactions as well. Let’s try it!

To lock a record, we need two fields:

  1. one to remember ID of agent that is processing this record (let’s call this column ‘locked_by‘)
  2. one another to know the time when the lock becomes obsolete (let’s call this column ‘expires_at‘)

I wrote a  class that inherits from Zend_Db_Table and helps to get records with locking them.

<?php

class Koodix_Db_Table_Lockable extends Zend_Db_Table
{
    protected $_lockedByField = 'locked_by';
    protected $_expiresAtField = 'expires_at';
    protected $_TTL = '1 HOUR'; //time to live for lock
    
    public function fetchLocked( Zend_Db_Table_Select $select, 
        $lockerID ) {
        
        $db = $this->getAdapter();
        $db->beginTransaction();
        
        $column = $db->quoteIdentifier( $this->_lockedByField );
        $select->forUpdate()
             ->where("$column=? OR $column IS NULL", $lockerID);


        $data = $this->fetchAll($select);
        if( empty($data) ) return null;
        
        $expiresAt = new Zend_Db_Expr('DATE_ADD( NOW(), 
            INTERVAL ' . $this->_TTL . ')');
        if( sizeof($this->_primary) > 1 ) {
            foreach( $data as $item ) {
                $item->{$this->_lockedByField} = $lockerID;
                $item->{$this->_expiresAtField} = $expiresAt;
                    
                $item->save();
            }
        }
        else {
            $arrIds = array();
            foreach( $data as $item ) {
                $arrIds[] = $item->id;
            }
            
            $this->update(
                array(
                    $this->_lockedByField => $lockerID,
                    $this->_expiresAtField => $expiresAt,
                ), 
                $db->quoteIdentifier(current($this->_primary)) . 
                    ' IN ("'.implode('","', $arrIds).'")'
            );
        }
        
        $db->commit();
        return $data;
    }

    public function releaseLocks( ) {
        
        $column = $db->quoteIdentifier( $this->_expiresAtField );
        
        return $this->update(
            array(
                $this->_lockedByField => null,
                $this->_expiresAtField => null,
            ), 
            "$column <= NOW()"
        );
        ;
    }
}

If the table has a composite primary key (containing more than one column), the ActiveRecord approach is used, so the save() method for every record is called, that’s simple (drawback — multiple update queries). Otherwise, if it is a deep-seated table with one ID column as a primary key, then the IDs are collected in a list and all records are updated by a single statement with IN in where clause (which is much faster).

TTL (‘Time to Live‘) — period of time when lock is allowed. In my application the default is one hour. Format of TTL can be seen in MySQL documentation.

And now how to use it.

Let’s imagine you have several editors that divide the big articles list and review them. My model class has a method fetchForUser() that returns no more than 5 articles for current user (by given user ID).

This is an Article table model, inherited from the class above. Usually such classes are located at

application/default/models/ArticleTable.php
<?php
class ArticleTable extends Koodix_Db_Table_Lockable
{
    protected $_name = 'article';
    
    public function fetchForUser( $userId, $count=5 ) {
        
        $select = $this->select()
            ->where('reviewed = 0')
            ->order('expires_at DESC')
            ->order('date_imported DESC')
            ->limit( $count );
        
        return $this->fetchLocked($select, $userId);
    }
}

Note: if the editor refreses the page, the expres_at fields is refreshed by current time as well.

As for step four of our algorithm (releasing all obsolete locks) — create an action in your backend controller, call your table model releaseLocks() method in it and call that action periodically by Cron.

To boost the performance of the lock releasing, create an index on the expires_at column. (Because of this reason I rejected the ‘locked_since‘ column in favor of ‘expires_at‘)

P.S. In my database date/time columns have DATETIME type. If you use INT to store timestamps, convert it to unix time and back.

Several sites on single WordPress installation

development, ideas, php, wordpress 5 Comments »

I have a couple of other WordPress blogs on the same server besides this one. One day I realised that all of them have 3 different WP versions and, as result, different admin areas which is not handy. I decided to make them use the same WordPress installation.

Ok, first of all, I deleted wp-admin and wp-includes folders and created new ones as symbolic links. Though the frontend worked well, I couldn’t log in into admin area, because the browser was redirected to that blog which was the base for all the rest for unknown reason.

The investigation shown, that admin area of WordPress is a separate sub-application, thing in itself, and in my case it resolves the absolute path to its source as the path to the base blog. I wanted each blog to use its own folder because there are config file and uploads folder.

It took me some time to find a solution. It requires two steps.

First, I added this line to the top of the .htaccess to make any PHP request to the blog (the blog front-end and the admin area scripts) call the same script before thier start:

#fix for several sites on the same WP installation
php_value auto_prepend_file "/var/www/site_doc_root/prepend.php"

In this code /var/www/ is the root folder for all my sites, and the site_doc_root is the document root of the current site (folder where all its files are located).

OK, the 2nd step — the contents of the prepend.php script. It is easy — it just must define an absolute path constant which is used all around the WordPress:

<?php
define('ABSPATH', dirname(__FILE__).'/');

OK, after that I decided not to use one of the blogs as source for others, but download a fresh copy of WordPress and make it a source of the symbolic links for all my blogs. This helps to update them.

Then I deleted wp-admin and wp-includes folders and some wp-files and recreated them as symlinks. Attention to wp-config.php — don’t delete it, keep it unique for every site!

To make this task easier, I created setup.sh file, pasted the contents I show below, run this command

chmod 755 setup.sh

then I copied it in every site folder and launched there for every site:


ln -s /var/www/wordpress/wp-admin wp-admin
ln -s /var/www/wordpress/wp-includes wp-includes

ln -s /var/www/wordpress/wp-app.php wp-app.php
ln -s /var/www/wordpress/wp-atom.php wp-atom.php
ln -s /var/www/wordpress/wp-blog-header.php wp-blog-header.php
ln -s /var/www/wordpress/wp-comments-post.php wp-comments-post.php
ln -s /var/www/wordpress/wp-commentsrss2.php wp-commentsrss2.php
ln -s /var/www/wordpress/wp-config-sample.php wp-config-sample.php
ln -s /var/www/wordpress/wp-cron.php wp-cron.php
ln -s /var/www/wordpress/wp-feed.php wp-feed.php
ln -s /var/www/wordpress/wp-links-opml.php wp-links-opml.php
ln -s /var/www/wordpress/wp-load.php wp-load.php
ln -s /var/www/wordpress/wp-login.php wp-login.php
ln -s /var/www/wordpress/wp-mail.php wp-mail.php
ln -s /var/www/wordpress/wp-pass.php wp-pass.php
ln -s /var/www/wordpress/wp-rdf.php wp-rdf.php
ln -s /var/www/wordpress/wp-register.php wp-register.php
ln -s /var/www/wordpress/wp-rss.php wp-rss.php
ln -s /var/www/wordpress/wp-rss2.php wp-rss2.php
ln -s /var/www/wordpress/wp-settings.php wp-settings.php
ln -s /var/www/wordpress/wp-trackback.php wp-trackback.php
ln -s /var/www/wordpress/xmlrpc.php xmlrpc.php

That’s not all 😉

I decided to update my WordPress installation every one or two months.

To do that, in the /var/www/ folder (where all my sites reside) let’s create an update script update_wordpress.sh with the following contents:

wget --timestamping http://wordpress.org/latest.zip
unzip -o latest.zip

This will download a fresh copy of the wordpress if it’s changed (though wordpress team doesn’t show the file Last-Modified header, I think one day they will) and unzip it to /var/www/wordpress/ folder which is the source for our symlinks.

Yes, you got it right — launching this script is all I need to update all my blogs.

Let’s make it periodic:

crontab -e

and then add this line to run the update process automatically every 1st day of every month at 9 AM:

0 9 1 * * /var/www/update_wordpress.sh > mail -s "Wordpress updated" your@email.com

P.S. Of course, SVN checkout can be used for that purpose 🙂

RSS feed of error log

development, ideas, php Comments Off on RSS feed of error log

Your PHP application logs every error to error log. Do you want to keep track of them in your favourite Feed Reader? Then follow up reading this article!

First of all, you must tell to your application to log errors to a log file. Two parameters must be set: what to log and where to save it. First can be done by error_reporting setting, second — by setting error_log PHP value:

error_reporting( E_ALL & ~E_NOTICE ); //all except notices
ini_set( 'error_log', 'temp/error.log' );

Put it in the beginning of your application, for example, at the top of index.php file.

By the way, you can write something custom in the error log by calling error_log function:

error_log('something');

OK, now, how to make a feed from your error log.

You can do this in a few ways.

First, let’s do it manually.

Manual solution

1. Remove ini_set setting described above. Then create .htaccess file in the top folder of your application and place this line in it:

php_value error_log temp/error.log

It will tell to all scripts of your application to take this setting into consideration without manual calling of ini_set function.

2. Create feed.php file. Place this line inside it:

$contents = file_get_contents( ini_get( 'error_log' ) );
echo nl2br( $contents );

It’s dirty for now, but enough as a check.

3. Make sure that your error log file is not empty. Now you can access your feed.php file via browser and check that it’s showing you the contents of your error log.

4. Now you should create an RSS feed from contents of the file. You can rely on your framework or use a custom solution, e.g. download EasyRSS class and fill the contents of the feed with data from the error log (you would need to think about regexp to parse date and error text from it).

After that you could feed your feed reader with the feed address ;]

Ready-made solution

Download a ready PHP class that will do all it for you — RSS Feed. Yes, it’s simple. You can even protect your feed with a password.

Notes:

  • on the production server change your error_reporting value, for example, to log fatal errors only.
  • log errors in the catch() part of the exception handling mechanism:
    try{ ... }
    catch(Exception $e) {
      error_log( $e->getMessage() );
    }
WP Theme & Icons by N.Design Studio
Entries RSS Comments RSS Log in