Eirik Hoem on his blog provides an overview of PHP’s register_shutdown_function, and suggests using it for the cases when for whatever reason your Web page ran out of memory, fatal’ed, and you don’t want to display a blank page to the users.
register_shutdown_function is also useful for command-line scripts with PHP. Pretty frequently your script has to do some task like parse a large XML file, and the test examples when it was originally written did not account for the XML file possible being huge. Therefore your script dies with like 23% completion, and you’re left with 23% of the XML file parsed. Not ideal, but a quick duct-tape-style fix, would be to introduce a register_shutdown_function call to system(), to which you pass the script itself.
If you happen to keep track of which line you’re on while parsing, you can pass the line number as the first parameter to your own script, and make it start off after that 23% mark, or wherever it died. The script then needs to be launched with 0 passed as the first parameter. It will run out of memory, die, launch register_shutdown_function, which will launch another copy of the script (while successfully shutting down the original process) with a new line number, which will repeat the process.
Again, this is a duct tape approach to PHP memory consumption issues while working with large data sets.
Saturday, November 8, 2008
register_shutdown_function possible use cases
24 Web site performance tips
Yahoo! Developer Network blog had an entry by Stoyan Stefanov and presentation from PHP Quebec conference. A few points to take away, in case you don’t feel like going through 76-slide presentation:
1. A drop of 100ms in page rendering time leads to 10% in sales on Amazon. A drop of 500 ms leads to 20% less traffic to Google.
2. Make fewer HTTP requests - combine CSS and JS files into single downloads. Minify both JS and CSS.
3. Combine images into CSS sprites.
4. Bring static content closer to the users. That usually means CDNs like Akamai or Limelight, but sometimes a co-location facility or data center in a foreign country is the only option.
5. Static content should have Expires: headers way into the future, so that they’re never re-requested.
6. Dynamic content should have Cache Control: header.
7. Offer content gzip’ed.
8. Stoyan claims nothing will be rendered in the browser till the last piece of CSS has been served, and therefore it’s critical to send CSS as early in the process as possible. I happen to have a document with CSS declared at the very end, and disagree with this statement - at least the content seems to render OK without CSS, and then self-corrects when CSS finally loads.
9. Move the scripts all the way to the bottom to avoid the download block - Stoyan’s example shows placing the javascript includes right before and , although it’s possible to place them even further down (well, you’d break XHTML purity, I suppose, if you declare your documents to be XHTML).
10. Avoid CSS expressions.
11. Consider placing the minified CSS and JS files on separate servers to fight browser’s default pipelining settings - not everybody has FasterFox or tweaked pipeline settings.
12. For super-popular pages consider inlining JS for fewer HTTP requests.
13. Even though placing content on external servers with different domains will help you with HTTP pipelining, don’t go crazy with various domains - they all require DNS lookups.
14. Every 301 redirect is a wasted HTTP request.
15. For busy backend servers consider PHP’s flush().
16. Use GET over POST any time you have a choice.
17. Analyze your cookies - large number of them could substantially increase the number of TCP packets.
18. For faster JavaScript and DOM parsing, reduce the number of DOM elements.
19. document.getElementByTagName(’*').length will give you the number of total elements. Look at those abusive
20. Any missing JS file is a significant performance penalty - the browser will browse the 404 page you generate, trying to see if it has valid
FirePHP for PHP and AJAX development
FirePHP is a package consisting of a Firefox extension and server-side PHP library for quick PHP development on top of Firebug. It allows you to include the PHP library, and issue logging calls like
fb('Log message' ,FirePHP::LOG);
fb(’Info message’ ,FirePHP::INFO);
fb(’Warn message’ ,FirePHP::WARN);
fb(’Error message’,FirePHP::ERROR);
This is visible only to the Firefox version that has FirePHP installed on top of Firebug. You can also dump entire array and objects to the fb() function call, and have them displayed in Firebug UI.
PHP Top scalability mistakes
1. Define the scalability goals for your application. If you don’t know how many requests you’re shooting for, you don’t know whether you’ve built something that works, and how long it’s going to last you.
2. Measure everything. CPU usage, memory usage, disk I/O, network I/O, requests per second, with the last one being the most important. If you don’t know the baseline, you don’t know whether you’ve improved.
3. Design your database with scalability in mind. Assume you’ll have to implement replication.
John Coggeshall, CTO of Automotive Computer Services, and author of Zend PHP Certification Practice Book and PHP5 Unleashed, gave a talk at OSCON 2008 on top 10 scalability mistakes. I wasn’t there, but he posted the slides for everybody to follow. Here’re some lessons learned.
1. Define the scalability goals for your application. If you don’t know how many requests you’re shooting for, you don’t know whether you’ve built something that works, and how long it’s going to last you.
2. Measure everything. CPU usage, memory usage, disk I/O, network I/O, requests per second, with the last one being the most important. If you don’t know the baseline, you don’t know whether you’ve improved.
3. Design your database with scalability in mind. Assume you’ll have to implement replication.
4. Do not rely on NFS for code sharing on a server farm. It’s slow and it’s got locking issues. While the idea of keeping one copy of code, and letting the rest of the servers load them via NFS might seem very convenient, it doesn’t work in practice. Stick to some tried practices like rsync. Keep the code local to the machine serving it, even if it means a longer push process.
5. Play around with I/O buffers. If you’ve got tons of memory, play with TCP buffer size - your defaults are likely to be set conservatively. See your tax dollars at work and use this Linux TCP Tuning guide. If your site is written in PHP, use output buffering functions.
6. Use Ram Disks for any data that’s disposable. But you do need a lot of available RAM lying around.
7. Optimize bandwidth consumption by enabling compression via mod_deflate, setting zlib.put_compression value to true for PHP sites, or Tidy content reduction for PHP+Tidy sites.
8. Confugure PHP for speed. Turn off the following: register_globals, auto_globals_jit, magic_quotes_gpc, expose_php, register_argc_argv, always_populate_raw_post_data, session.use_trans_sid, session.auto_start. Set session.gc_divisor to 10,000, output_buffering to 4096, in John’s example.
9. Do not use blocking I/O, such as reading another remote page via curl. Make all the calls non-blocking, otherwise the wait is something you can’t really optimize against. Rely on background scripts to pull down the data necessary for processing the request.
10. Don’t underestimate caching. If a page is cached for 5 minutes, and you get even 10 requests per second for a given page, that’s 3,000 requests your database doesn’t have to process.
11. Consider PHP op-code cache. This will be available to you off-the-shelf with PHP6.
12. For content sites consider taking static stuff out of dynamic context. Let’s say you run a content site, where the article content remains the same, while the rest of the page is personalized for each user, as it has My Articles section, and so on. Instead of getting everything dynamically from the DB, consider generating yet another PHP file on the first request, where the article text would be stored in raw HTML, and dynamic data pulled for logged-in users. This way the generated PHP file will only pull out the data that’s actually dynamic.
13. Pay great attention to database design. Learn indexes and know how to use them properly. InnoDB outperforms MyISAM in almost all contexts, but doesn’t do full-text searching. (Use sphinx if your search needs get out of control.)
14. Design PHP applications in an abstract way, so that the app never needs to know the IP address of the MySQL server. Something like ‘mysql-writer-db’, and ‘mysql-reader-db’ will be perfectly ok for a PHP app.
15. Run external scripts monitoring the system health. Have the scripts change the HOSTS if things get out of control.
16. Do not do database connectivity decision-making in PHP. Don’t spend time doing fallbacks if your primary DB is down. Consider running MySQL Proxy for simplifying DB connectivity issues.
17. For super-fast reads consider SQLite. But don’t forget that it’s horrible with writes.
18. Use Keepalive properly. Use it when both static and dynamic files are served off the same server, and you can control the timeouts, so that a bunch of Keep-alive requests don’t overwhelm your system. John’s rule? No Keep-alive request should last more than 10 seconds.
19. Monitor via familiar Linux commands. Such as iostat and vmstat. The iostat command is used for monitoring system input/output device loading by observing the time the devices are active in relation to their average transfer rates. The iostat command generates reports that can be used to change system configuration to better balance the input/output load between physical disks. vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.
20. Make sure you’re logging relevant information right away. Otherwise debugging issues is going to get tricky.
21. Prioritize your optimizations. Optimization by 50% of the code that runs on 2% of the pages will result in 1% total improvement. Optimizing 10% of the code that runs on 80% of the pages results in 8% overall improvement.
22. Use profilers. They draw pretty graphs, they’re generally easy to use.
23. Keep track of your system performance. Keep a spreadsheet of some common stats you’re tracking, so that you can authoritatively say how much of performance gain you got by getting a faster CPU, installing extra RAM, or upgrading your Linux kernel.
Sphinx search ported to PECL
Anthony Dovgal reported on adding open source SQL full-text search engine sphinx to PECL. The documentation is available on the PHP site, the engine is available upon including sphinxapi.php in your application.
You know the usual InnoDB vs. the MyISAM trade-offs, where the former is faster, but the latter has the full-text search? Sphinx is a free open-source full-text search engine that works with many RDMBS, and now is pretty easy to incorporate into PHP. A simple example of calling Sphinx is available here:
$s = new SphinxClient;
$s->setServer("localhost", 6712);
$s->setMatchMode(SPH_MATCH_ANY);
$s->setMaxQueryTime(3);
$result = $s->query("test");
Best practices in PHP development
1. Use source control
1. First, choose between distributed and non-distributed
2. Then, if you chose non-distributed, choose between CVS and SVN
3. In Subversion, use trunk/ for ongoing development and bug fixes, branches/ for ongoing large projects that later need to be merged in, and tags/ for releases
4. Use svn externals to connect to remote repositories
5. Subversion supports pre-commit and post-commit hooks for better code maintainability and checks
2. Implement coding standards
1. Develop class, variable, function, package, etc. naming conventions
2. Agree on common formatting as far as spacing, braces, etc.
3. Implement comment standards
4. PHP_CodeSniffer can run on pre-commit to check whether the commit adheres to the standards
5. Don’t forget to enforce coding standards on any outsourced projects
3. Unit testing and code coverage
1. Use PHPUnit for unit testing
2. For continuous integration, check out phpUnderControl
3. For integration testing, check out Selenium, a general Web application testing suite
4. Documentation
1. Don’t invent your own standards, see what phpDocumentor has to offer. Doxygen also supports phpDoc tags
2. For documenting the software project, try DocBook - XML-based format that allows you to quickly publish a PDF document, or a Website with documentation
5. Deployment
1. Have a standard deployment process that a rookie can familiarize with quickly
2. Support 3 environments - development, staging, and production
3. Deploy code only from repository tags, don’t run trunk, or allow code editing on server
4. Check out a new tag from SVN, point the symlink to it. If something goes wrong during release, change the symlink back to the previous version - easy rollback strategy
5. Everything that needs to be done on the production servers needs to be automated
6. You can do another Selenium test after the release is deployed
7. Check out Monit and Supervisord for deployment monitoring
12 PHP optimization tips
1. If a method can be static, declare it static. Speed improvement is by a factor of 4.
2. Avoid magic like __get, __set, __autoload
3. require_once() is expensive
4. Use full paths in includes and requires, less time spent on resolving the OS paths.
5. If you need to find out the time when the script started executing, $_SERVER['REQUEST_TIME'] is preferred to time()
6. See if you can use strncasecmp, strpbrk and stripos instead of regex
7. str_replace is faster than preg_replace, but strtr is faster than str_replace by a factor of 4
8. If the function, such as string replacement function, accepts both arrays and single characters as arguments, and if your argument list is not too long, consider writing a few redundant replacement statements, passing one character at a time, instead of one line of code that accepts arrays as search and replace arguments.
9. Error suppression with @ is very slow.
10. $row['id'] is 7 times faster than $row[id]
11. Error messages are expensive
12. Do not use functions inside of for loop, such as for ($x=0; $x < count($array); $x) The count() function gets called each time.
Online Forms? Get Vizzual!
VizzualForms is a Web-based data management system created for collecting, storing and processing data via the Internet. It allows you to create and publish Web forms in mere minutes without any programming knowledge or advanced computer skills.
It is a time- and cost-effective solution for private or business use. It is very easy to use, yet it is powerful. It includes a visual form creator, data processing tool, contact manager, invitation module and report tool.
Company Name
Vizzual
The idea of VizzualForms is to allow users to create even the most complicated forms in a simple manner. It is about collecting, storing and processing data.
CEO’s Pitch
VizzualForms is a Web-based data management system created for collecting, storing and processing data via the Internet. It allows you to create and publish Web forms in mere minutes without any programming knowledge or advanced computer skills.
It is a time- and cost-effective solution for private or business use. It is very easy to use, yet it is powerful. It includes a visual form creator, data processing tool, contact manager, invitation module and report tool.
With VizzualForms users can create a broad spectrum of forms, invite others to fill in the forms and see or share the results in real time. Our integrated email notification system can send email with data to a mailbox each time a form is submitted, guaranteeing that no request or data will be missed. Forms created with VizzualForms can be embedded into Web page or delivered using an invitation module.
Mashable’s Take
There are a number of applications and services, online and off, that will accomplish the task of producing forms for numerous purposes. Tools featured within the near-ubiquitous Microsoft Office can manage things like surveys and registration forms and whatnot. The same with Apple’s iWork suite and other similar releases. Meanwhile, Web-based options like FormSite, ExtremeForm, JotForm, and Wufoo can accomplish needed designs with varying degrees of customization and interactivity.
Vizzual Forms is one such character. Users of Vizzual Forms are served with what basically amount to two utilities. One intended for form assembly, the other for data analysis. The first is fairly intuitive to operate. It generally works the way the average user might expect it to. Its strongest features are for editing. Sending things to a user’s contact list might be a little tedious at first, but so long as you maintain an account with regularity, you’ll learn the proverbial ropes pretty quickly, we think. Besides, if you’re conducting an online poll or making some other request, embeds can be easily made.
Depending on your level of engagement, you can make do with a cost-free and ad-supported account, or spring for a Basic ($8.50), Small Biz ($16.50), or Enterprise-level registration ($31.50). Companies looking to host Vizzual on their own servers can even pony up $1,900 for the privilege.
To be honest, what Vizzual Forms will help you produce won’t be aesthetically brilliant by any means. The examples it reveals are pretty ordinary. Apart from a palette of colors, you might say things are a bit boring. But rarely is the act of filling a form with information a truly fun endeavor. So the important thing, then, is for the service to help Joe Creator ease through the process as best he or she can. And Vizzual Forms appears conditioned to do just that. You can certainly be specific with an editing job if you like. Nothing really stops you from spending a great deal of time fine-tuning a poll or whathaveyou. But you’ll rarely, if ever, be lost for a visual guide on how to get it done.
Monday, November 3, 2008
Understanding a Robots.txt File
The robots.txt file allows you to control the behavour of web crawlers and spiders that visit your site. Most web crawlers are harmless and simply collect data for various reasons like search engine listings, internet archiving, validating links, security scanning, etc. It's always a good idea to create a robots.txt to tell the crawlers where they can go and where they can not.
A crawler should always follow the "The Robots Exclusion Protocol" and therefore whever it comes to a web site to crawl it, it first checks the robots.txt file.
www.yourdomain.com/robots.txt
Once it has processed the robots.txt file it will then proceed to the rest of your site usually starting at the index file and traversing throughout. There are quite often places on a web site which do not need to be crawled, like the images directory, data directories, etc so these are what you need to place into your robots file.
The "/robots.txt" file is simply a text file, which contains one or more records. A single record looking like this: User-agent: *
Disallow: /
The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the web site.
A basic tobots.txt example User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /junk/
Allowing a single crawler User-agent: Google
Disallow:
User-agent: *
Disallow: /
To exclude a single robot
User-agent: BadBotRead More......
Disallow: /
People-search sites Reunion.com, Wink to merge
Social network Reunion.com has made a new friend: people search service Wink. The two have merged in a new deal that promises to make it dramatically easier to find people on the Web.Early next year, the merger will produce "an entirely new brand," the companies said. The two have not said what its name will be, nor have financial details been disclosed. With the dual technologies of Reunion and Wink, the companies say that they will be able to search more than 700 million social-networking profiles. They'll be able to search profiles on MySpace, Facebook, LinkedIn, Friendster, AOL's Bebo, Microsoft's Windows Live Spaces, Yahoo, Xanga, and Twitter--among others.
Numbers from Nielsen last month indicated that Reunion.com, which says it receives 12 million unique visitors each month, is one of the fastest-growing social networks in the U.S. despite the fact that it's hardly on the radar of Twittering blog pundits. Its biggest demographic, according to Nielsen, is those between 55 and 64 who are looking to re-connect with friends and classmates.
"Through this merger, we're redefining the people search space by bridging existing social networks and providing consumers with the tools they need to find, be found, and stay connected," Wink CEO Michael Tanne said in a release. "We're aiming to create an entirely new online experience that simplifies people's lives by making it easy to find and keep up with everyone they know. There will be exciting developments in the coming months as we integrate our strengths and push our business forward."
News Source
