Wednesday, December 01, 2010

SQLite and Doctrine: in memory databases

Recently for a client's project, I incorporated a SQLite in memory database into my persistence tests as its a good (and fast) alternative to managing separate unit test and system databases. Typically in my test setup / tear down methods, I drop and recreate the unit test database to ensure clean testing conditions.... however I found that Doctrine doesn't support SQLite in memory database drops.

To drop an in memory db - I don't believe the standard sql drop statement works. As this is the case, when calling Doctrine::dropDatabases(), Doctrine's SQLite Driver produces in a 'Database could not be found' Exception in its dropDatabases method.

As the SQLite in memory databases exist for the life of the connection only, a good solution is to simply reset the connection.

Burrowing through the Doctrine framework, within the SQLite Driver (Doctrine/Connection/SQLite.php) I made the following modification:


 public function dropDatabase()
    {
        if ( ! $dsn = $this->getOption('dsn')) {
            throw new Doctrine_Connection_Exception('You must create your Doctrine_Connection by using a valid Doctrine style dsn in order to use the create/drop database functionality');
        }
        
        $info = $this->getManager()->parseDsn($dsn);
        
        //
        // BENS EDIT: If the db is in memory only - simply recreate a new connection
        //
        if(strcasecmp($info["dsn"], "sqlite::memory:") == 0)
        {         
         $c = $this->getManager()->getCurrentConnection();

  $this->getManager()->closeConnection($c);
  $this->getManager()->connection("sqlite::memory:","unit_test");
     
         return;
        }
        //
        // END BENS EDIT
        //

        $this->export->dropDatabase($info['database']);
    }

There are probably better ways to do this - so feel free to leave comments - however the above worked for me...

Thursday, November 25, 2010

CodeIgniter and PHP Howto - Embedding images in Email

DOWNLOAD THE CODEIGNITER EMAIL EXTENSION HERE

I recently had an issue in a client's development which required images to be embedded in html email code (<img src='cid:embeddedImage' />), rather than referenced via a public url (i.e. <img src='http://www.mysite.com/myImage.png' />).





QUICK OVERVIEW - THE PUNCH LINE

I have written a CodeIgniter Library Extension to the basic Email class that facilitates the embedding of images in emails. You can download the code HERE.

To implement it, follow the instructions for Extending Native Libraries on the CodeIgniter website http://codeigniter.com/user_guide/general/creating_libraries.html

Finally, in the body of your email, use the following macro - making sure the class_id attribute matches the one used in the img src attribute (i.e. <img src='cid:my_image' />). An exmaple message body would be as follows


<html>
<body>
<img src="cid:my_image" />
</body>
</html>

// Macro in Windows
{embedded_image file=C:\\my_image.png class_id=my_image}{/embedded_image}

//Alternative Linux Macro
{embedded_image file=/var/my_image.png class_id=my_image}{/embedded_image}


And thats it - the library encodes the image file as a base64 string and embeds it in your email.

PROS AND CONS OF EMBEDDING IMAGES IN EMAIL:

PROS

- Images are immediately displayed when opening a message, rather than the client prompting the user to allow remote content... great for newsletters
- Emails are independent, once downloaded, they don't require a live connection to view in all their glory

CONS

- A little bit more coding and knowledge of email message format is required to get them to work if your framework does not support embedding images in email
- Some conjecture about spam filtering for bulk messages with embedded images. Some believe spam filtering is more pessimistic for emails with embedded images than not when sending to more than 100 addresses

HOW TO EMBED AN IMAGE

I'm not going to go too far into the coding of native php code to email with embedded images (unless I get requests) - so Ill give a brief overview and some references.

Like most things on the web, email message content can be defined in a series of envelopes. These envelopes are defined by content type. You can read about all of them here http://www.freesoft.org/CIE/RFC/1521/15.htm

One of the most useful images to exemplify the content structure of a html email can be found at http://www.phpeveryday.com/articles/PHP-Email-Using-Embedded-Images-in-HTML-Email-P113.html
Bit of a warning though - the article itself has buggy code and examples.

Grossly speaking - your email message if it supports html with embedded images, and a plain text alternative (for non html email clients) should render the following content in the email body



Content-Type: multipart/alternative; boundary="UNIQUE_ID_1"

--UNIQUE_ID_1
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

.... plain text alternative content for your html email....

--UNIQUE_ID_1
Content-Type: multipart/related;
 boundary="UNIQUE_ID_2"

--UNIQUE_ID_2
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

--UNIQUE_ID_2
Content-Type: image/jpeg
 name='embedded_image.jpg'
Content-Transfer-Encoding: base64
Content-ID: <class_id_referenced_in_img_src>
Content-Disposition: inline;
 filename='embedded_image.jpg'

(Base64 encoded binary for the image)

--UNIQUE_ID_2-- 

--UNIQUE_ID_1--


NOTE on the above.

-- means open content type. Trailing -- means closing content type. When content type is 'multipart', you can define more than one content type under the same section identifier as you are either specifying alternative content, or related content. multipart/alternative content means one of the specified content sections is used. multipart/related content means that one content section references the other. We use the multipart/alternative content type to specify html, and plain text message alternatives. We use the multipart/related for relating html content with embedded image content.

Monday, November 22, 2010

noscript tag and google index

The noscript tag is indexed by google! I recently released work for a client which used the noscript tag to display the typical 'this site needs javascript' warning if javascript was disabled on the client's browser.... unfortunately I did this in the head of my page AND used a h3 element to title the warning segment....

As a result, most pages on the site were indexed as 'Javascript not enabled' as the title, and the site's metatagged description as the content. Not happy.

Best practice - if you are going to use the noscript tag for simple javascript warnings, use it at the bottom of the rendered page.... many sites do this (including StackOverflow). Might also help not to use any heading elements (i.e. h1,h2,h3) but rely on styling another element as a heading.

Sunday, November 07, 2010

Free DNS Servers: Free DNS Online

For the last few years of web development, I have had to find a couple of reliable free DNS online services to manage my, and client Domains. I thought I'd post up a few and rate my experience with them.

Free DNS Online - My Experiences


Zone Edit : A great free DNS online service, 100% private and 100% free. Accounts used to be able to serve up to 5 domains, however recently this has been restricted to 2. The User Interface / Administration feels a bit clunky and old school web - but pretty straight forward and easy to use.

FreeDNS : Another good free DNS online service, however you entries can be reviewed by other users. Other users can configure subdomains off your domain, however you can pipe this through an authorisation process (where you the administrator are emailed and confirm any config changes). Payment is required if you want to make your listing private.

DNSExit : An uncapped free DNS online service. You can have as many accounts as you require... you can even set up dynamic DNS which is nice. Very slow though - and I have some questions regarding reliability. Recently the service was brought down by a DDOS attack for a couple of days.... have a backup - secondary dns on an alternate name server if possible.

DynDns : Free Dynamic DNS online service. Been around for years - can't recommend it enough for publicly serving from your dev box. Great for limited serving (i.e. demoing sites during development). Have a range of other paid services - however, a bit of googling for these will uncover alternative free services.


Tuesday, November 02, 2010

Detect Popup Blocker: Popup Blocker Detection for Chrome + all browsers

Popup blocker detection for all major browsers including Chrome....





Chrome popup blocker detection is a little different to others in that Chrome returns a valid window object after calling window.open, even with popups being disabled. There are a heap of posts around all basically stating that to determine whether this window object has been blocked by Chrome, we need to test whether the innerHeight of the popup has been set to 0.

Friday, October 15, 2010

Google Analytics - multi domain named site tracking



Another quick blog... Recently I added analytics to a couple of sites with more than one domain name. Its easy to see how much of your traffic is from each hostname (Vistors->Network Properties->Hostnames), but
I wanted to be able to overlay how much traffic was arriving to the site from each domain (to see which one was most effectively being advertised).....

I was able to achieve this by creating a Custom Segment.

When first logging into the Google Analytics Dashboard for the desired site / account, just above the Date Range, there is the Advanced Segments drop down list. Clicking on that drop down I was able to create a new Advanced Segment.

In the Advanced Segment editor, I could expand the Content submenu from the Dimensions menu on the left, and drag the 'HOSTNAME' field into my segment.

I then set the Condition field to 'CONTAINS', then enter the base hostname of the additional domain I want to analyse separately (i.e. myseconddomain.com).

After naming and saving the segment, I can apply it via the Advanced Segments drop down in the dashboard and viola....

Friday, October 08, 2010

CodeIgniter - Supporting Multiple Domains in config["base_url"]

A very quick blog.....

Within the site config file (application_folder/config/config.php file), a base_url property is set. This is read by the base_url() method to generate server side redirections......

If you are creating a site which has more than domain name (ie. www.domain_one.com and www.domain_two.com), its probably a good idea to dynamically create this value in the config file. This way, the domain name is preserved when redirecting between pages.


//...config ...//
$config['base_url'] = 'http';
if (isset($_SERVER["HTTPS"]) && $_SERVER["HTTPS"] == "on") $config['base_url'] .= "s";

$config['base_url'] .= "://";

if ($_SERVER["SERVER_PORT"] != "80") $config['base_url'] .= $_SERVER["SERVER_NAME"].":".$_SERVER["SERVER_PORT"];
else $config['base_url'] .= $_SERVER["SERVER_NAME"];

$config["base_url"]."/";
//... config ...//

Wednesday, October 06, 2010

IE Cache and AJAX: Cache Busting Ajax requests

Yet another 'special case' caveat for Internet Explorer, the red headed step child of the browser family..... (sorry to any red headed step children who might be reading this - chalk it up to the savage injustices of life). I discovered that IE cache and Ajax requests are not the best of friends compared to how other browsers handle Ajax requests.



Recently I found that IE cached ajax requests in a CodeIgniter + ExtJS site. I was using url rewriting, so all GET params were encoded as URI segments...

eg. (http://host/controller/action/param1/param2)

The Problem:
Usually, I would use ExtJS inherent cache busting tools (Ext.Ajax.disableCaching - which normally defaults to true).... but due to the url rewriting, the ExtJS method caused issues. Query string values (?blah=value) are disallowed in my app due to the url rewriting, therefore EXTs native cache disabling does not work as it simply appends a uid to the querystring (?_dc=123453443343). This caused 'disallowed character' exceptions.

Furthermore - I couldn't simply add a random variable to the end of a request, as this could be misinterpreted as an actual parameter for actions with parameters with default values

eg. http://host/controller/action/param1/param2/no_cache01223213312

no_cache01223213312 could be misinterpreted as param3 in the following action:
public function action($param1, $param2, $param3 = "default value")
{
//..//
}


The Solution:
The Big Stick:
Whether you use an MVC framework, URL rewriting, the first thing you should consider is that on all Ajax actions, make sure the header 'pragma' is set to no-cache..... so in php - write the header somewhere before content is returned to the browser

header("Pragma: no-cache");

This can really suck as it blows away all your lovely server side cache introducing a potential performance bottleneck to your app, all because Dwayne Dibley is still browsing your site using IE.

The ExtJS (Javascript) way:
The Ext solution was at the page level to intercept all AJAX requests, and add a random POSTED variable to the parameter listing.


Ext.Ajax.disableCaching = false;
Ext.Ajax.addListener("beforerequest", function (conn, options ){
   if(!options.params) options.params = {};
   options.params.cacheBuster = Ext.id();
}, this);



This forces a server side request as the request is unique (thanks to the random post variable). It also allows me to free specify GET params in the rewritten url, as I am adding a POST variable to uniquify the request.

For generic javascript.... when calling the target url, simply append a generated dummy query string parameter (like a timestamp)

"http://myhost.com/myAjaxPage.php?_nocache=321313213445462"

Again, the same caveat applies to the pragma header - you would probably want to make this cache busting parameter conditional on browser type

Saturday, October 02, 2010

Extra Bucks Online

So in the past I have blogged a bit about finding an eBusiness which is quick to setup and get running. I'm pleased to announce that my latest attempt at this is entering its final proofing.....

ExtraBucks is now online and is in its final proofing stages. The official launch date will come soon.... Essentially it is an odd jobs and piece work bulletin board. Its designed for those who want to make a bit of extra work outside of business hours to link up with people who need the odd job done. All 100% free to use. Check it out - have a play and feel free to leave some comments!

Monday, August 09, 2010

Calling the php interpreter within web script

Recently, part of a development required me to execute shell script as part of a web request. The design was to kick start a long running php process (via shell script), which would run beyond the length of a standard web request. To do this, I used forking shell script and popen... (if using shared hosting, shell execution methods are generally disabled - check php.ini's disable_functions config value to verify).

Within my dev environment (Windows) I was able to execute the following with no issues:

popen("start /b php my_script.php my_args", "w");
Note: 'start /b' forks a windows process - making it run in the background



When executing its equivalent in a Linux environment
popen("php my_script.php my_args &", "w");

In Linux, the server was thrown into a state of confusion, perpetually executing, then aborting the requested shell script command (infinitely starting and exiting the requested php script in the popen command). Obviously explicitly calling the php interpreter from within an executing php process is a particularly nasty thing to do.

I'm assuming this sort of thing doesn't happen in windows as shell script executes in its own command window (which when using proc_open with get_proc_status makes getting a correct PID value problematic). Anyone with futher insight - feel free to leave comments.

To get around this issue, I ensured that in the linux environment, the shebang path to the interpreter appeared on the first line of the executing script, and it was chmod'ed to allow direct cli execution... ie

my_script.php:
#!/usr/local/bin/php

//
// my code
//
?>

CHMOD command used:
chmod 755 my_script.php

Then finally the linux popen command:
popen("./my_script.php my_args &", "w");

The default working directory of popen, and all other shell script execution methods is always web root (so my_script.php sat in the web root dir).

The same issue occurs with all other shell script execution methods:
system, exec, proc_open, passthru, back ticks (`) etc.

Sunday, June 27, 2010

CodeIgniter - Ruby on Rails (RoR) Layouts and filters

DOWNLOAD THE CODEIGNITER DOCTRINE STARTER (~8Mb)

Continuing to list starter features, I have also integrated Ruby on Rails (RoR) style layouts and before / after filters.

Both were implemented as CI hooks - both the layout and filter code was obtained from the CodeIgniter forums and wiki posts

Using the Filters hook, each action is wrapped in a Doctrine Transaction, which ensures all db updates etc. are ATOMIC.

One hassle I found with the above Filters system was that I couldn't implement a before / after filter directly in the controller class. Within the Doctrine_Transaction filter I ensure that if a controller has a before_action, or after_action function within its class definition, it is called before, or after the action is executed. This obviously could / should be abstracted out eventually into its own Filter class.... but it works just the same as is.

Tuesday, June 08, 2010

CodeIgniter - Doctrine unit testing - starter features




So to expand on the previous post... specifically to describe a few of the extra bells and whistles in the project starter....


Unit Testing
I have created a (very) simple extension to the codeigniter unit testing as I wanted to have a simple, web based interface to unit tests. I found the whole CodeIgniter terminology used in unit testing a little confusing (coming from a NUnit / JUnit testing background). Unless I missed something in the doc (which is very possible), CodeIgniter supports test assertions ($controller->unit->run when the test helper is loaded), there was no way to group a number of assertions together into a logical unit of testing.

I'm going to use a little NUnit / JUnit terminology to describe unit testing via the web page interface in the starter. In the ApplicationCode/libraries folder I have a base_test_controller which when extended replicates basic TestFixture functionality. To create a Unit Test within the fixture, simply create a function prefixed with the name test_. The following demonstrates using a controller as a test fixture:



//
// My Test Fixture
//
class MyTest extends base_test_controller {
//
// This method is executed before any test methods are run
//
function fixture_setup(){}
//
// This method is executed after all test methods are run
//
function fixture_tear_down(){}
//
// This method is executed before each test method is run
//
function unit_setup(){}
//
// This method is executed after each test method is run
//
function unit_tear_down(){}

//
// Your Unit Test
//
function test_My_unit_test()
{
$this->test_description = "This is a description of my unit test";

$this->unit->run(true, true, "Assertion one has passed");
$this->unit->run(true, true, "Assertion two has passed");
}
}




Saturday, June 05, 2010

PHP, CodeIgniter, Doctrine and ExtJS


So, its been ages since I have posted anything on this particular blog, so time for an update.
-->


Although I still dabble with MS technologies, I find the bulk of my development has moved to MVC geared frameworks (ideally rails based) - which (I find) works best within the contexts of dynamic languages..... so I have been using platforms like Ruby, JRuby, PHP, even ColdFusion for the last few projects. My tech stack of choice is:

ExtJS : client side JS library
JRuby : presentation tier
Java + Hibernate : domain + data layer

In the latest project however I have been developing a survey site in PHP. I thought that to save anyone else time, I have created a 'project starter' development stub which incorporates:
  • ExtJS 3.2.0: client side JS library,
  • CodeIgniter 1.7.2: presentation + domain tier,
  • Doctrine 1.2: ORM data layer framework

The idea being that you can simply extract and develop, without wasting time integrating all the techologies so that they work together....

Some features of this project starter are:
  • CodeIgniter Models have been replaced with Doctrine Records
  • Doctrine is loaded into CI as a plugin
  • RoR type before and after filters....
  • Doctrine transactions automatically wrapped around every action at execution time (ATOMIC db updates)
  • Basic Role based security (I think Redux may be in there as well?)
Simply extract, hook up the database.php config file and viola.... You can start coding your layouts, views and models.

Licensing: refer to all integrated frameworks licensing agreements for details (i.e. Doctrine, ExtJS, CodeIgniter). Original licensing agreements are bundled in the Project starter for your reference. The Project starter is a freely distributed, open source integration of technologies. Just make sure that you are using each framework in adherence to their respective licensing terms. As far as I am aware all technologies can be freely used under GPL....