Web browsers: Y u no autosave textarea?

Web browsers are still missing a fundamental feature: autosaving textarea input.

It should function like any other editor that we’re used to. Microsoft Word has autosaving and so does this WordPress content area when drafting a post. Even the single line input fields remember what you’ve previously submitted them with.

Why are browsers still missing this feature as we use them more and more as actual apps? Gmail? Google Docs?

You can build in the auto-save ability with Javascript such as this WordPress content area but why do developers have to build and manage this?

If there doesn’t already exist, I’m proposing an “HTML5″ addition to autosave textareas into “LocalStorage” and provide an API for Javascript to easily interact with it. I’m thinking <textarea name="blog_content" autosave></textarea>

I should be able to right click (Windows) in the textarea and select which revision to update it to or delete all entries if I wish.

Worthy idea or terrible?

Posted in Standards, Web | Tagged , , , , , | Leave a comment

Symfony2 + Varnish-like ESI with Akamai ETS mod_esi Edge Side Include

Akamai provides their own version of an ESI testing server found here:

http://www.akamai.com/html/support/esi.html

Of course it’s not open source, it’s a pre-compiled binary and “Caching of template and fragments is not supported”. Who would shoot themselves in the foot by providing a free version of their commercial offering? Ok yes, Red Hat does but moving on…

I grabbed the FC7 version for CentOS 5.5, extracted, ran the install script with an ETS listening on port 80 and origin host at localhost and port 81. Turn on ESI Debugging too, it’s rather informative being able to view the source with ESI tags and the actions of the server on those tags. To see the debug info view the page source of a request since they’re hidden.

Edit the standard Apache install at /etc/httpd/conf/httpd.conf and change it to listen on port 81.

After setting up Symfony 2′s native ESI handling demo as detailed here:

http://symfony.com/doc/current/book/http_cache.html#edge-side-includes

Akamai’s config will need a couple headers in order for Symfony to respond with ESI tags in the response.

Edit the Akamai config here: /usr/local/ETS/conf/httpd.conf

Edit the VirtualHost directive near the end of the file and add these two directives:

SurrogateHeader Surrogate-Capability ESI/1.0
Accept-Esi 1.0

Now Symfony 2 will see that an ESI Surrogate is making a request and it will respond with the ESI tag instead of filling it in. Develop your app using port 81 and test your ESI through port 80. View the page source to see the debugging info if debugging is enabled.

This guide is similar to the cookbook entry for Varnish which also requires that the surrogate header be set:

http://symfony.com/doc/current/cookbook/cache/varnish.html

Posted in CentOS, Linux, PHP, Server, Symfony2 | Tagged , , , , | 1 Comment

Simple Cloud – Error Handling Not Portable

In my previous article about the lack of exceptions in PHP libraries, I came across an inconsistency with Simple Cloud‘s API (now known as the Zend_Cloud component) between file storage adapters.

The issue is that Windows Azure and Nirvanix storage adapters will always throw an exception on an unsuccessful attempt (checked against the API response) whereas Amazon S3 adapter will return true/false (checked against API response) or an exception (some other underlying client error) therefore breaking portability. You could add an explicit check for false but that isn’t obvious. There’s also no use of the @throws tag in the docblock making this even less obvious.

I’ve only checked this against the file storage part of Simple Cloud but I seem to remember the same no-exception issue being in the Amazon SQS API as well.

Zend\Service\Amazon\S3.php – putObject()

$response = $this->_makeRequest('PUT', $object, null, $headers, $data);

// Check the MD5 Etag returned by S3 against and MD5 of the buffer
if ($response->getStatus() == 200) {
    // It is escaped by double quotes for some reason
    $etag = str_replace('"', '', $response->getHeader('Etag'));

    if (is_resource($data) || $etag == md5($data)) {
        return true;
    }
}

return false;

Zend\Service\WindowsAzure\Storage\Blob.php – putBlob() to putBlobData()

// Perform request
$response = $this->_performRequest($resourceName, '', Zend_Http_Client::PUT, $headers, false, $data, Zend_Service_WindowsAzure_Storage::RESOURCE_BLOB, Zend_Service_WindowsAzure_Credentials_CredentialsAbstract::PERMISSION_WRITE);
if ($response->isSuccessful()) {
    //isSuccessful checks for 2xx or 1xx status code
    return new Zend_Service_WindowsAzure_Storage_BlobInstance();
    //stripped code for blog article
} else {
    throw new Zend_Service_WindowsAzure_Exception($this->_getErrorMessage($response, 'Resource could not be accessed.'));
}

Another issue is that the docblock for Zend\Cloud\StorageService\Adapter\S3.php storeItem() has “@return void” which isn’t the case (most of the other methods appear to be this way as well).

Related bug report without attention: http://framework.zend.com/issues/browse/ZF-9436

Posted in Amazon Web Services, Cloud, PHP, Zend Framework | Tagged , , , , | 1 Comment

Lack of Exceptions in PHP Libraries

Why are new PHP libraries still failing to fully utilize exceptions? Is it PHP4 compatibility? Speed? Maintain the API across exception-less languages such as C? Simply because most developers still don’t effectively use exceptions?

It’s a shame that PHP exceptions are still not being employed in recent libraries despite exceptions were initially announced over 5 years ago. A recent and frustrating experience was with researching and working with PHP libraries for Amazon S3.

We could go into the semantics of “exception” & “error” and when they should be thrown but let’s skip that and sum it simply as this: If the library is not able to fully complete the requested action then it needs to throw an exception detailing why.

In other words, if the library cannot save my image to Amazon S3 then it needs to throw an exception with Amazon’s error message. If it cannot delete my image it needs to throw an exception. If it cannot update the meta-data of my image it needs to throw an exception.

I researched three libraries:

None of them throw an exception on error with Amazon S3. The only exceptions thrown are for validation like the bucket name or maybe some lower error in the Zend HTTP request class.

For each of these I researched into the “put/create object” method:

Zend Framework’s Amazon S3: Only returns true or false. I was forgiving though because that piece of the library was probably designed earlier in the framework and so maintained API. I also was having trouble actually getting the an error message or the entire response object as bug reported http://framework.zend.com/issues/browse/ZF-9436

Simple Cloud: Uses the Amazon S3 library of Zend Framework and simply returns the return value of it ie. true or false. (This is actually a problem I found with Simple Cloud’s portability with other adaptors)

AWS SDK for PHP: Returns a CFResponse object that requires a isOK() method check.

I do not want to test if the library call was successful or not in some C-like true/false or some PHP4-style PEAR::isError() manner. I do not want to test for “false” and then have to ask the library for the error. I want to consolidate my error handling; use exceptions in your libraries please.

I understand it’s unfair to group all libraries into this article when this article is only about AWS S3 libraries, even though there are other recent libraries that still don’t use exceptions, but this is mainly to promote awareness.

Posted in Amazon Web Services, Cloud, PHP, Zend Framework | Tagged , , , | 2 Comments

AWS Route 53 – Anycast Routing Performance

Update 2010-12-25: AWS techs informed me that Route 53 is designed with a balance between latency and availability hence why not all name servers go to the nearest data center like I was expecting. Another AWS tech informed me that DNS resolvers utilize various performance techniques such as issuing parallel queries and remembering which nameserver responded fastest. Although I do not know of any resolvers that actually implement these techniques, and the AWS tech has yet to respond, I think this is a solid decision for Route 53 and the tests following are an example of that.

I ran a few tests from some of the datacenters I have servers in (should’ve ran some tests from the East coast..). They don’t impress me that much and raise a concern for when Route 53 adds location-aware results because I sure don’t want to send Northern California users all the way to the Europe (Ireland) region. Hopefully they’re working out these kinks although this is a routing level issue and possibly out of their control.

Here are the results based on the a rob.olmos.name hosted zone:

Assigned:

ns-1860.awsdns-40.co.uk
ns-1206.awsdns-22.org
ns-77.awsdns-09.com
ns-937.awsdns-53.net

Ping from Torrance, CA (Covad):

ns-1860.awsdns-40.co.uk 72ms
ns-1206.awsdns-22.org 7ms
ns-77.awsdns-09.com 40ms
ns-937.awsdns-53.net 7ms

Only two using LA datacenter..

Ping from Torrance, CA (Verizon FiOS):

ns-1860.awsdns-40.co.uk 80ms
ns-1206.awsdns-22.org 12ms
ns-77.awsdns-09.com 49ms
ns-937.awsdns-53.net 12ms

Only two using LA datacenter…

Ping from LA datacenter:

ns-1860.awsdns-40.co.uk 67ms
ns-1206.awsdns-22.org <1ms
ns-77.awsdns-09.com 30ms
ns-937.awsdns-53.net <1ms

Not all using the LA datacenter…

Ping from DFW Rackspace datacenter:

ns-1860.awsdns-40.co.uk 30ms
ns-1206.awsdns-22.org 2ms
ns-77.awsdns-09.com 59ms
ns-937.awsdns-53.net 36ms

Only one using the DFW datacenter?

Ping from Chicago Rackspace datacenter:

ns-1860.awsdns-40.co.uk 23ms
ns-1206.awsdns-22.org 23ms
ns-77.awsdns-09.com 7ms
ns-937.awsdns-53.net 34ms

Only one using the St. Louis datacenter

Ping from EC2 West A:

ns-1860.awsdns-40.co.uk 150ms
ns-1206.awsdns-22.org 150ms
ns-77.awsdns-09.com 21ms
ns-937.awsdns-53.net 21ms

Two of them going to Germany from Northern California???

Sample dig queries return similar response times.

Posted in Amazon Web Services, Cloud, Web | Tagged , , , | 1 Comment

Why Amazon Route 53 (DNS) is Important in AWS

The other night I was trying to finish up a blog post on why AWS needs to add DNS to their services and then I get an email announcing Route 53, AWS’s DNS service. About time. Instead of saying why AWS needs it as I originally was going to, I’ll detail why this is a very important addition, which covers the same ground.

AWS is big on CNAMEs. Any place you want a “pretty domain” (or smaller too if you’re trying to cut down on the HTML response size) instead of AWS’s monster URLs you have to use a CNAME. CNAMEs work fine and it’s nice that AWS supports them for services like CloudFront and S3 (Virtual Hosting) as compared to other cloud providers… With the Elastic Load Balancing it’s required because the IP address changes. Usage of CNAMEs with AWS have a couple subtle issues, though, two of them being management and latency.

The management isn’t ultimately as big of an issue as the latency issues. Unless you need an API to the DNS, multiple identifies/accounts to manage the records, and so on then management isn’t much of a problem besides being managed by another provider.

The latency, however, can be a bigger issue. When CNAMEs are used with DNS records they don’t actually provide an answer. What they actually provide is a more of a finger-point that says “look over here for the answer.” This results in extra look-ups and extra look-ups are overhead that add latency to the requests.

Amazon’s DNS and Route 53, which is currently on UltraDNS and will eventually moved to Route 53, is rather fast because it is anycast’ed. Anycast allows servers in multiple datacenter’s to handle the request which suits UDP well since it is not connection oriented.

However, it’s likely the DNS servers for your domain isn’t anycast. It’s also possible the DNS servers aren’t geographically close to your target audience if your target audience is in a specific region such as Los Angeles. If you’re really unlucky, the randomly assigned DNS servers from the registrar could be on the east coast with your website servers and target audience on the west coast (assuming the US or anywhere). That’s latency even the fastest website servers cannot eliminate. It’s not terrible, but it’s also not as good as it could be nowadays.

Another pitfall with CNAMEs is that some (most? all? Network Solutions does) DNS servers will return the root servers in the authority and additional sections of the DNS answer pushing the UDP packet near the size limit. Keeping in mind how long the AWS CNAMEs can be too eg. my-elb-65147120.us-west-1.e lb.amazonaws.com this gets very close.

You will want to avoid exceeding the UDP packet size limit at all possible because when that happens the whole DNS resolution has to happen again over TCP (unless the resolver & server make use of the the extension mechanism for DNS EDNS) which has even more overhead than UDP. That can potentially be an added 150ms of simple DNS resolution free of charge. Not good.

Route 53 to the rescue. When AWS integrates the Elastic Load Balancer into Route 53 next year, this and zone apex redirect and hopefully the CNAME business will be done away with. Because only AWS knows what IP your ELB is currently operating as they can fill that IP into whichever record you designate avoiding the CNAME and avoiding the subsequent DNS resolutions and latency. Maybe they’ll integrate CloudFront and S3 and also do away with those CNAMEs. Combine that with the anycast low-latency, high-availability benefits of Route 53 and that is one sweet boost.

Posted in Amazon Web Services, Cloud, Server, Web | Tagged , , , | 9 Comments

FTP Protocol – Why Two Ports and Connections?

I’m finding that Stack Overflow and Server Fault fuel me with motivation to actually dig up answers to questions I’ve had but was too busy or could not easily find the answer to.

One of the questions I’ve had is why does the FTP protocol have an unconventional usage of two ports when other protocols are able to easily do all their business over one port and one connection.

“sims” on Server Fault also had the same question. I spent some time researching and found some info as to why:

http://serverfault.com/questions/202813/why-was-the-ftp-protocol-designed-to-use-more-than-one-port

Posted in Server, Web | Leave a comment

Logitech Mouse Back/Forward Diamond/Spade Firefox Find

http://forums.logitech.com/t5/Mice-and-Trackballs/Firefox-Quickfind/m-p/386805

Set them to “Other” > “Generic button” … Who would’ve thought?

The silliest part of it all is I went to Logitech’s site first instead of Google..

Posted in Uncategorized | Leave a comment

CentOS 5.5 + PHP 5.3.3 + PHP-FPM + APC + nginx (yum RPMs)

(2011) Update: IUS renamed the packages to have a suffix of “u”. This was done to avoid conflicts with CentOS’s recently released PHP 5.3 packages. I prefer IUS since it maintains PHP’s version numbers and there’s some handy PECL extensions.
(2010-12-03) Update: EPEL updated nginx to 0.8.53 around early November. This post has been updated to use that package instead of IUS 0.7.
(2010-09-22) Update: IUS has fixed the repo: https://bugs.launchpad.net/ius/+bug/645409
(2010-09-22) Update: IUS updated the PEAR and APC packages and their repo appears to have come out of sync so you’ll experience errors until it’s fixed. Download the packages manually or use APC from repo 5 and FPM from repo 5.5 while making sure to “yum clean all” in between.
(2010-09-16) Update: I completely forgot that PHP-FPM also solves the APC cache issues that were present with php-fcgi where every php-fcgi process had it’s own APC cache. I include those steps.

IUS Community recently published PHP 5.3.3 PHP-FPM package but it isn’t in the default repo (which appears to be CentOS 5.4 and not 5.5). If you cannot install php53u-fpm then try changing the repo as detailed here:
https://bugs.launchpad.net/ius/+bug/591609

This post assumes that PHP is not already installed. If it is you’ll possibly get conflicts with some of the PHP packages when installing the IUS ones. IUS has a “yum replace” script but I have not used it. Otherwise, uninstall the old PHP RPMs before proceeding.

Install EPEL and IUS-Community repo packages:

#EPEL
wget http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/epel-release-1-1.ius.el5.noarch.rpm
#IUS Community
wget http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/ius-release-1.0-6.ius.el5.noarch.rpm
#Install repos
rpm -Uvh epel-release-1-1.ius.el5.noarch.rpm ius-release-1.0-6.ius.el5.noarch.rpm

You’ll then need to edit the ius repo to force it to CentOS 5.5 (check if this is still necessary by trying the next step first):

vi /etc/yum.repos.d/ius.repo
#Comment out this line:
mirrorlist=http://dmirr.iuscommunity.org/mirrorlist?repo=ius-el5&arch=$basearch
#Add this line:
baseurl=http://dl.iuscommunity.org/pub/ius/stable/Redhat/5.5/$basearch

Now you can easily install PHP 5.3.x, the new PHP-FPM package, and APC (it works correctly with PHP-FPM, yay) via:

#IUS PHP 5.3 packages are prefixed with "php53u" to avoid name clashes
yum install php53u-fpm php53u-pecl-apc
#Accept the dependencies

Next install nginx from IUS. Note that this is nginx 0.7.65 so it appears to have the usual backports from 0.8.

Next install nginx 0.8.53 from EPEL.

#Use EPEL nginx. See directly below if you want IUS 0.7
yum install nginx
#Note that IUS's nginx is suffixed with "07"
#yum install nginx07

Now you have the necessary packages installed. They don’t Just Work(TM) together out of the box so you’ll need to do some minor config changes.

For nginx edit /etc/nginx/nginx.conf and uncomment the entire block shown below to enable fastcgi forwarding for .php requests:

#location ~ \.php$ {
#    root           html;
#    fastcgi_pass   127.0.0.1:9000;
#    fastcgi_index  index.php;
#    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
#    include        fastcgi_params;
#}

Within that same block also change:

fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;

To:

fastcgi_param  SCRIPT_FILENAME  /usr/share/nginx/html$fastcgi_script_name;

Now make a phpinfo to test:

#mmm phpinfo
echo "<?php phpinfo();" > /usr/share/nginx/html/phpinfo.php
#make sure APC is running. then copy the stats
cp /usr/share/doc/php53-pecl-apc-3.1.3p1/apc.php /usr/share/nginx/html/

Start it up:

chkconfig php-fpm on
chkconfig nginx on
service php-fpm start
service nginx start

Try it all out in a browser.

Note that this was the least number of steps to get this stack going. As usual be sure to check the other config settings for performance and security, like nginx’s number of connections and workers and changing php-fpm’s default user/group from the default of apache and chroot it to a jail.

Posted in CentOS, PHP | Tagged , , , , | 67 Comments

Neo4j REST API and PHP

I was trying out Neo4j due to my curiosity of the graph specialization. Although Neo4j is not designed to run stand-alone like a database server there is a sub-project that adds a REST API to allow non-Java applications to make use of Neo4j. Neo4j is pretty simple, you basically have nodes, relationships, and properties on both. That’s about it.

In this post is a very basic hypothetical scenario (mainly about Neo4j) of a digital posting system where:

  • Users can have a relationship with other users
  • Users can post messages
  • Users can hide other users’s messages

Please be aware that I have not thoroughly tested this.

Just so you have a visual idea of what it looks like inside Neo4j I wrote a simple graph via Graphviz. Green = KNOWS, Blue = POSTED, and Red = HIDES.

Here’s some snippets of code (that should really be cleaned up to more reflect the image but I know you will understand) to create some nodes and relationships using the Neo4J-REST-PHP-API-client library. This is minimal to maintain clarity so feel free to add more users, messages, and relationships.

<?php
//client setup
require('php-neo-rest.php');
$graphDb = new GraphDatabaseService('http://localhost:9999/');
$uri = 'http://localhost:9999/node/1/traverse/node';

//User_A
$f1 = $graphDb->createNode();
$f1->user_id = 1;
$f1->save();

//UserB
$f2 = $graphDb->createNode();
$f2->user_id = 2;
$f2->save();

//Post_B_1
$f2m1 = $graphDb->createNode();
$f2m1->message = 'f2 post';
$f2m1->timestamp = time();
$f2m1->save();

//User_A knows User_B
$f1rf2 = $f1->createRelationshipTo($f2, 'KNOWS');
$f1rf2->save();
//User_B posting Post_B_1
$f1rm2 = $f1->createRelationshipTo($f1m2, 'POSTED');
$f1rm2->save();

//User_A hiding User_B's Post_B_1
$f1rf2m1 = $f1->createRelationshipTo($f2m1, 'HIDE');
$f1rf2m1->save();

And finally, the traversal:

$traverse = array(
	"order" => "breadth first",
	"prune evaluator" => array(
    	"language" => "builtin",
    	"name" => "none"
	),
	"return filter" => array(
    	"language" => "javascript",
    	"body" => "(function shouldReturn(pos) {
    		var node = pos.node();
    		if(!node.hasProperty('message')) {
    			return false;
    		}
    		var rels = node.getRelationships();
    		var relIterator = rels.iterator();
    		var rel = null;
    		while(relIterator.hasNext()) {
    			rel = relIterator.next();
    			if( (rel.getType() == 'HIDE' || rel.getType() == 'POSTED')
    				&& rel.getStartNode().getId() == 1) {
    				return false;
    			}
    		}
    		return true;
    	})(position);"
	)
   	//"max depth" => "1"
);
var_dump(HTTPUtil::jsonPostRequest($uri, $traverse));

The Neo4j REST component embeds the Rhino engine so the traversal is actually written in javascript. The “return filter” function is only passed a position object of the traversal’s current position. Other objects can be retrieved such as the current node or last traversed relationship via this variable. Only the nodes that return true in the “return filter” are returned from the traversal.

This traversal can be paraphrased as “find UserA’s (identified by an ID of 1) friend’s message postings that have not been hidden by UserA”. We’re only interested in messages so skip (by returning false) any nodes without a “message” property. We then get all the relationships on that node, grab an iterator to loop over the relationships, and skip it if that node has been posted or hidden by UserA.

Posted in Neo4j, PHP | Tagged , , | 5 Comments