whenpenguinsattack.com

Thursday, March 30, 2006

PHP editor bonanza

By Justin Silverton

The following is a list of PHP editors (commercial and freeware), with reviewed links to php-editors.com:

Editor Name Version License Platform/OS Our Rating User Rating
PHP Edit 1.2.5 Freeware
Shareware
Commercial
Other
Windows
5/5 4.55
(125 votes)
Dreamweaver 8 Commercial
Windows
5/5 3.98
(58 votes)
NuSphere PhpED 4 Commercial
Windows
Linux
5/5 4.27
(73 votes)
Maguma Workbench 2.6 Commercial
Windows
Linux
Mac
5/5 4.11
(9 votes)
emacs 21 Freeware
Windows
Unix
Linux
Mac
Other
5/5 4.07
(13 votes)
ActiveState Komodo 3.5 Commercial
Other
Windows
Unix
Linux
Other
5/5 4.57
(21 votes)
PHP Designer 2005 3.0.6 Freeware
Windows
5/5 4.74
(2329 votes)
Komodo 3.1 Shareware
Commercial
Other
Windows
Linux
Other
5/5 4.13
(22 votes)
TSW WebCoder 2005 2005 Commercial
Other
Windows
5/5 2
(4 votes)
VIM 6.1 Freeware
Windows
Unix
Linux
4/5 4.23
(146 votes)
DzSoft PHP Editor 1.4 Shareware
Commercial
Windows
4/5 4.54
(22 votes)
Davor's PHP Constructor 1.0 Shareware
Commercial
Windows
4/5 3.75
(4 votes)
Edit Plus 2.11 SR-2 Shareware
Commercial
Windows
4/5 4.67
(80 votes)
HTML-Kit 292 Freeware
Commercial
Windows
4/5 4.48
(43 votes)
PHP Expert Editor 2.5 Shareware
Commercial
Windows
4/5 4.39
(81 votes)
Anjuta 1.0.1 Freeware
Unix
Linux
4/5 3.66
(6 votes)
Bluefish 0.12 Other
Linux
4/5 3.97
(34 votes)
Quanta Plus 3.2.1 Freeware
Linux
4/5 4.59
(61 votes)
Zend Studio 5 Commercial
Windows
Unix
Linux
Mac
Other
4/5 3.93
(143 votes)
Kate 2.2 Freeware
Linux
4/5 4.11
(18 votes)
Maguma Studio Free 1.1.0 Freeware
Windows
4/5 3.89
(47 votes)
PHP Editor by EngInSite 3 Shareware
Commercial
Windows
4/5 3
(4 votes)
PHP Eclipse 1.06a Freeware
Unix
Linux
4/5 4.28
(70 votes)
Xored:: WebStudio 0.3.4 Freeware
Windows
Unix
Linux
Other
4/5 3.92
(25 votes)
SciTE 1.53 Freeware
Windows
Unix
Linux
Other
4/5 4.57
(56 votes)
VS.Php Beta 3 Commercial
Other
Windows
4/5 4.57
(19 votes)
Maguma Studio Pro 1.3.X Commercial
Windows
4/5 4
(8 votes)
Macromedia HomeSite 5.5 Commercial
Windows
4/5 4.5
(14 votes)
TextPad 4.7.2 Freeware
Commercial
Windows
4/5 4.51
(27 votes)
PHP Edit 1.0 Stable Freeware
Shareware
Commercial
Windows
4/5 4.44
(78 votes)
EngInSite Editor for PHP 2.2 Shareware
Commercial
Windows
4/5 4.31
(22 votes)
BBedit 7.0 Commercial
Mac
3/5 4.47
(17 votes)
BBedit Lite 6.1 Freeware
Mac
3/5 4.55
(9 votes)
Cooledit 3.17.7

3/5 3
(2 votes)
Nedit 5.3 Freeware
Unix
Linux
3/5 4.57
(7 votes)
PSPad 4.3.0 Freeware
Windows
3/5 4.67
(76 votes)
PHP Coder 3 Freeware
Windows
3/5 3.89
(64 votes)
AceHTML Pro 5 Shareware
Commercial
Windows
3/5 4
(10 votes)
Top PHP Studio v1.19.6 Shareware
Commercial
Windows
3/5 4.33
(6 votes)
jEdit 4.1 Freeware
Windows
Unix
Linux
Mac
Other
3/5 4.62
(72 votes)
SubEthaEdit 1.1.5 Freeware
Mac
3/5 4.5
(10 votes)
umdev 2004 Shareware
Windows
3/5 2.33
(3 votes)
Dev-PHP 3.0 Freeware
Windows
3/5 4.48
(56 votes)
Crimson Editor 3.60 Freeware
Windows
3/5 4.70
(47 votes)
PHP Processor 1.2 Shareware
Windows
3/5 5
(1 votes)
tsWebEditor 2 Freeware
Other
Windows
3/5 4.61
(13 votes)
Svoi.NET - PHP Edit XP 4.0 Freeware
Windows
3/5 4.16
(18 votes)
ConTEXT 0.97.4 Freeware
Windows
3/5 4.70
(24 votes)
PHP Side (Simple IDE) 0.4 Freeware
Windows
Unix
Linux
3/5 4.07
(13 votes)
HAPedit 3.1 Freeware
Windows
3/5 4.44
(18 votes)
EmEditor 4.0 Freeware
Shareware
Windows
3/5 4
(4 votes)
Roadsend Studio 1.1.1 Commercial
Windows
Unix
Linux
3/5 1
(1 votes)
TruStudio 1.0.0. Freeware
Windows
Unix
Linux
Mac
3/5 3.33
(9 votes)
Smultron 1.0.1 Freeware
Mac
3/5 4.28
(7 votes)
PHP backend generator 0.9 Commercial
Windows
Unix
Linux
Mac
Other
3/5 1
(1 votes)
PHPMaker 3.2 Shareware
Commercial
Windows
3/5 5
(1 votes)
Pidela 0.1 Freeware
Windows
Unix
Linux
Mac
3/5 1
(1 votes)
Arisesoft Winsyntax 2 Freeware
Windows
2/5 4.12
(25 votes)
SEG 1.0.1 Freeware
Windows
2/5 4.46
(13 votes)

The early days of slashdot

By Justin Silverton

I have been a regular reader (and occasional poster) of the popular tech site http://www.slashdot.org since 2000. Recently, I searched archive.org for the earliest version of the site I could find.

Here is a link: http://web.archive.org/web/19980113191222/http://slashdot.org/
it is slashdot.org on January 13th, 1998

Some interesting archived article summaries taken from the above link:

IE Takes the Lead?

Contributed by CmdrTaco on Fri Jan 09 at 2:09PM EST From the fun-with-numbers deptDavid Fagan wrote in to tell us about this article where it is reported than recent statistics show that IE4 is out on top in the browser battle with 63% of the traffic at various high traffic sites. I don't put a lot of weight in these stats, but this is a pretty significant number.

Intel Releases 266 Pentium

Contributed by CmdrTaco on Fri Jan 09 at 12:29PM EST From the pushing-but-not-hard deptIntel is releasing the 266Mhz version of the possibly immortal Pentium line of CPUs. Supposedly this chip is primarily for the Laptop world. Interesting timing considering how hard Intel is pushing the P2 lines of chips, and the next generation of that line that is due out soon. Thanks to David Fagan for alerting us.

(a direct link to an article posted by cmdr taco about the current state slashdot):

http://web.archive.org/web/19980113194013/slashdot.org/slashdot.cgi?
mode=article&artnum=00000425

Monday, March 27, 2006

Flickr.com - PHP/mysql case study

Introduction

Carl Henderson from Flickr.com, a very popular photo blogging service has released a pdf (not sure exactly when this was actually released) detailing the issues they faced with having a high-traffic website.


original PDF can be downloaded here

some interesting points taken from this pdf are below.

Classes, libraries, and systems used

1) smarty for templating
2) PEAR and XML for Email parsing
3) perl for controlling
4) imagemagick for image processing
5) mysql (4.0/innoDb)
6) java, for node service
7) apache 2 and redhat linux

8) 60,000 lines of PHP code
9) 60,000 lines of templates
10) 70 custom smarty functions/modifiers
11) 25,000 DB transactions/second at peak
12) 1000 pages per second at peak

unicode support

1) UTF-8 pages
2) CJKV support

Tips: don't use HtmlEntities(). Also, Javascript has patchy Unicode Support

Why php was used

1) Everything can be stored in the database, including smarty cache
2) a "shared nothing" approach (as long as php sessions were not used)

Mysql usage

Select's: 44,220,588
Insert's: 1,349,234
update's: 1,755,503
delete's: 318,439
13 select's per Insert, Update, and delete

Tips: many of the tables that needed to be full-text searched were de-normalized. This does waste space, but because it allowed for little or no joins, it made searching much faster.

Friday, March 24, 2006

protecting your PHP code


By Justin Silverton

Introduction

A client of mine approached me today and was interested in releasing a PHP based product, but didn't want his source code to be viewed, in plaintext, by the people purchasing it (mainly because competitors can could easily just purchase a copy and integrate his source code into their product). So, I researched the different options available to protect source code.

What doesn't work

The various encoders available do not work. These companies/products should just release these products as accelerators (which can improve speed by up to 10X) and not a secure and reliable way of hiding source code.

http://www.phprecovery.com is a website that charges money to decode the following types of encoded files (it is just an example site that I found. There are many more just like it):
  • Zend
  • Ioncube
  • SourceGuardian
  • TurckMM
  • SourceCop
  • ScopBin
  • Zend (Gaspra)
  • Ioncube (last)
  • CodeLock
This site has been tested and it does work. Most people would not bother with the hassle of paying someone to decode your application, but if you offer a more expensive version that includes the full source (and the price is more than it would cost to decode it), then it might just be a better solution.

What works

The best solution is code obfuscation. It may not be perfect, and in some instances, you may have to change your code around a little bit, but it will make it very difficult to re-use your source code.

I prefer a free program called POBS, available Here

How it works:

Replace names

POBS replaces user-defined (NOT predefined) functions, constants and variables with a MD5 key of 8 characters. (It doesn't use MD5 keys of 32 bytes, which is standard, since that would increase the size of your sourcecode). 8 bytes seems enough to give each functions or variable its unique name. MD5 is not reversible.

The first letter of the new functionname is a "F", of a variable a "V" and of a constant a "C"

The function with name MakeImageHtml is replaced by Fee2c1bdc
The variable $ImgText is replaced by $V1d9d94a6
The constant USERDIR is replaced by C389a367e

Futher obscuring

In addition, POBS can be instructed to concatenate lines and remove comments and indents. These are not irreversible since a person can write a program to add indents and returns. But it really makes a mess of your code and therefore furtherly discourages many wouldbe hackers from trying to reverse-engineer your code.

Exclude stuff

POBS allows you to indicate which user-defined variables, constants and functions need to be excluded from replacing. In the settings file "pobs-ini.inc" you can add these names to the arrays $UdExVarArray, $UdExcConstArray and $UdExcFuncArray. Do NOT use dollarsigns here.

In $UdExVarArray you are allowed to use wildcards in the form of an asterix (*) at the end of each variablename. I.e. params_* will exclude params_type, params_address and params_name. So if you name your variables to a certain convention you can easily and securely exclude them by group. This way you don't have to be afraid you forgot to add it to the array in case you added a new variable to your code.

Process

POBS consists of 2 major processes.

1. POBS first scans all the files with the file-extensions allowed in the sourcedirectory. While scanning, it makes a list of userdefined variables, functions and constants it has located in your sourcecode

2. POBS now knows which ones it should replace and starts writing new files in the target directory

Tuesday, March 21, 2006

The Zend Framework


By Justin Silverton


Introduction

The Zend Framework is a recently released (still in alpha) set of open source tools for php designed for developing Applications and Web Services.

Included Functionality

Zend_Controller and Zend_View

These components provide the base for a simple MVC website and are already used on this site and several others. A front controller dispatches requests to page controllers. It is as minimalist as possible and we're working to make it even simpler. The Zend_View component provides encapsulation for view logic. It can use templates written in PHP or can be combined with a third-party template engine.

Zend_Db

Database access is a very light layer on top of PDO. Solutions existing systems not using PDO (such as mysqli or oci8) are presently under development. Included are adapters, a profiler, a tool to assist with building everyday SELECT statements, and simple objects for working with table row data.

Zend_Feed

The links on the sidebars of our home page are generated using Zend_Feed. This component provides a very simple way to consume RSS and Atom data from feeds. It also includes utilities for discovering feed links, importing feeds from different sources, and feeds can even be modified and saved back as valid XML.

Zend_HttpClient

This component provides a client for the HTTP protocol and does not require any PHP extensions. It drives our web services components. In time, we will develop support for extension-based backends such as cURL.

Zend_InputFilter

The input filtering component encourages the development of secure websites by providing the basic tools necessary for input filtering and validation.

Zend_Json

Easily convert PHP structures into JSON for use in AJAX-enabled applications.

Zend_Log

Log data to the console, flat files, or a database. Its no-frills, simple, procedural API reduces the hassle of logging to one line and is perfect for cron jobs and error logs.

Zend_Mail and Zend_Mime

Almost every internet application needs to send email. Zend_Mail, assisted by Zend_Mime, creates email messages and sends them. It supports attachements and does all the MIME dirty work.

Zend_Pdf

Portable Document Format (PDF) from Adobe is the de facto standard for cross-platform rich documents. Now, PHP applications can create PDF documents on the fly, without the need to call utilities from the shell, depend on PHP extensions, or pay licensing fees. Zend_PDF can even modify existing PDF documents. Create a sharp customer invoice in Adobe Photoshop, fill in the order from Zend_Pdf, and send it with Zend_Mail.

Zend_Search_Lucene

The Apache Lucene engine is a powerful, feature-rich Java search engine that is flexible about document storage and supports many complex query types. Zend_Search_Lucene is a port of this engine written entirely in PHP 5, allowing PHP-powered websites to leverage powerful search capabilities without the need for web services or Java. Zend_Search_Lucene's file format is fully binary compatible with its Java counterpart.

Zend_Service: Amazon, Flickr, and Yahoo!

Web services are becoming increasingly important to the PHP developer as mashups and composite applications become the standard for next generation web applications. The Zend Framework provides wrappers for service APIs from three major providers to make the as simple to use as possible. We're working on more and engaging API vendors directly to make PHP the premier platform for consuming web services.

Zend_XmlRpc

PHP 5's SOAP extension dramatically lowered the bar for communicating with SOAP services from PHP. Zend_XmlRpc brings the same capabilities to XML-RPC, mimmicking the SOAP extension and making these services easier to use than ever from PHP 5.

Conclusion

The Zend Framework looks promising, but I think that in its current state, it is more of a set of classes than an actual framework. Currently, PEAR is a much better choice in terms of community support and component availability. I am glad that Zend is continuing to embrace the open source community and I will be curious to see the future builds of this framework.

Download

It can be downloaded Here

Sunday, March 19, 2006

php security mistakes - part 2

By Justin Silverton

In one of my previous articles, I mentioned the top 5 security mistakes made in PHP. This article is a follow-up, with some more common security mistakes.

System Calls

In PHP, there are different ways to execute system calls. The system(), exec(), and passthru() all allow you to execute operating-system commands from within your scripts.

Each of these functions, if not checked, can also allow a malicious user to exploit your system and execute commands that could possible access private files and information.

Protecting your system from this attack

The input from the user, no matter the context, should never be trusted. PHP provides two functions, escapeshellarg() and escapeshellcmd().

The escapeshellarg() function is designed to remove or otherwise eliminate any potentially harmful characters received from user input for use as arguments to system commands (in our case, the zip command).

The syntax for this function is as follows:escapeshellarg($command)
where $command is the input to clean, and the return value is the cleaned string. When executed, this function will add single quotes around the string and escape (add a slash in front of) any single quotes that exist in the string.

escapeshellcmd() is similar to this function, except it will only escape characters that have a special meaning to the underlying operating system. If user input will be used as part of the argument list for a system call, the escapeshellarg() function is always the better choice.

File Uploads

PHP will create a file with the uploaded content, but will not check whether the filename is valid, or if the type and size are correct

A user could potentially create his own form specifying the name of some other file that contains sensitive information and submit it, resulting in the processing of that other file.

Solution

use move_uploaded_file() or is_uploaded_file(). However, there are some other problems with user-uploaded files and check the $_FILES super global array to make sure that the user has uploaded the correct file type/size.


Including Files

In PHP you can include local or remote files by using include(), include_once(), require() and require_once(). It allows you to have separate files for classes, reused code and so on, increasing the maintainability and readability of your code.

The concept of including remote files is dangerous in itself, though, because the remote site could be compromised or the network connection could be spoofed. In either scenario, you are injecting unknown and possibly hostile code directly into your script.

Another issue to think about when including files, is if a file that is included is dependent on user input. This poses a potential securty issue, which can be fixed by verifying and cleaning incoming varialbes.

Conclusion

Don't trust any incoming variables ($_GET,$_POST, or $_COOKIE). These can all be set by a malicious user and possibly compromise the securty of your system.

Wednesday, March 15, 2006

php template engine roundup

By Justin Silverton

After searching the Internet for a good template engine, I have compiled the following list:

Smarty
Smarty is a template engine that compiles the templates into PHP scripts, then executes those scripts. Very fast, very flexible.

Heyes Template Class
A very easy to use, yet powerful and quick template engine that enables you to separate your page layout and design from your code.

FastTemplate
A simple variable interpolation template class that parses your templates for variables and spits out HTML with their values

ShellPage
A simple and easy to use class that lets you make whole websites based on template files for layouts. Change the template and your whole site changes.

STP Simple Template Parser
A simple, light weight and easy to use template parser class. It can assemble a page from several templates, output result pages to the browser or write them to the filesystem.

OO Template Class
An object oriented template class you can use in your own programs.

SimpleTemplate
A template engine to create and structure websites and applications. It can translate and compile the templates.

bTemplate
A small and fast template class that allows you to separate your PHP logic from your HTML presentation code.

Savant
A powerful but lightweight PEAR-compliant template system. It is non-compiling, and uses PHP itself as its template language.

ETS - easy template system
A template system that allows you to reshuffle templates with exactly the same data.

EasyTemplatePHP
A simple, yet powerful templating system for your site.

vlibTemplate
A fast, full featured template system that includes a caching and debugging class.

AvanTemplate
A template engine that is multi-byte safe and consumes little computing resource. It supports variable replacement and content blocks that can be set to hidden or shown.

Grafx Software's Fast Template
A modification of the popular Fast Template system, this includes a cache function, debug console, and silent removal of unassigned dynamic blocks.

TemplatePower
A fast, simple and powerful template class. Features nested dynamic block support, block/file include support and show/hide unassigned variables.

TagTemplate
This library function was designed for use with template files and allows you to retrieve info from HTML files.

htmltmpl: templating engine
A templating engine for Python and PHP. Targeted to web application developers, who want to separate program code and design of their projects.

PHP Class for Parsing Dreamweaver templates
A simple class to parse a Dreamweaver template for use in custom mods for a Gallery 2 and a WordPress blog.

MiniTemplator (Template Engine)
A compact template engine for HTML files. It features a simple syntax for template variables and blocks. Blocks can be nested.

Layout Solution
Simplifies website development and maintenance. It holds commonly used variables and page elements so you don't need to duplicate common layouts over and over.

Cached Fast Template
This inclusion into FastTemplate allows for caching of the template files, and can even cache with different specifications on separate blocks of content.

TinyButStrong
A template engine that supports MySQL, Odbc, Sql-Server and ADODB. It includes seven methods and two properties.

Brian Lozier's php based template engine
Only 2k in size, very fast and object-orientated.

WACT
a template engine that separates code from design.

PHPTAL
a XML/XHTML template library for PHP.

Tuesday, March 14, 2006

10 database speed tests

By Justin Silverton

I came across the following 10 benchmark tests covering:

SQLite version 3.3.3
SQLite version 3.3.3
SQLite version 2.8.17
SQLite version 2.8.17
PostgreSQL version 8.1.2
MySQL version 5.0.18
FirebirdSQL version 1.5.2

About the hardware/database settings used:

All databases were installed with default settings.
Tests were run on 1.6GHz Sempron with 1GB of ram and 7200rpm SATA disk running Windows 2000 + SP4 with all updates applied.

Test 1: 1000 INSERTs

CREATE TABLE t1(a INTEGER, b INTEGER, c VARCHAR(100));INSERT INTO t1 VALUES(1,13153,'thirteen thousand one hundred fifty three');INSERT INTO t1 VALUES(2,75560,'seventy five thousand five hundred sixty');... 995 lines omittedINSERT INTO t1 VALUES(998,66289,'sixty six thousand two hundred eighty nine');INSERT INTO t1 VALUES(999,24322,'twenty four thousand three hundred twenty two');INSERT INTO t1 VALUES(1000,94142,'ninety four thousand one hundred forty two');

SQLite 3.3.3 (sync):
3.823
SQLite 3.3.3 (nosync):
1.668
SQLite 2.8.17 (sync):
4.245
SQLite 2.8.17 (nosync):
1.743
PostgreSQL 8.1.2:
4.922
MySQL 5.0.18 (sync):
2.647
MySQL 5.0.18 (nosync):
0.329
FirebirdSQL 1.5.2:
0.320

Test 2: 25000 INSERTs in a transaction

BEGIN;CREATE TABLE t2(a INTEGER, b INTEGER, c VARCHAR(100));INSERT INTO t2 VALUES(1,298361,'two hundred ninety eight thousand three hundred sixty one');... 24997 lines omittedINSERT INTO t2 VALUES(24999,447847,'four hundred forty seven thousand eight hundred forty seven');INSERT INTO t2 VALUES(25000,473330,'four hundred seventy three thousand three hundred thirty');COMMIT;

SQLite 3.3.3 (sync):
0.764
SQLite 3.3.3 (nosync):
0.748
SQLite 2.8.17 (sync):
0.698
SQLite 2.8.17 (nosync):
0.663
PostgreSQL 8.1.2:
16.454
MySQL 5.0.18 (sync):
7.833
MySQL 5.0.18 (nosync):
7.038
FirebirdSQL 1.5.2:
4.280

Test 3: 25000 INSERTs into an indexed table

BEGIN;CREATE TABLE t3(a INTEGER, b INTEGER, c VARCHAR(100));CREATE INDEX i3 ON t3(c);... 24998 lines omittedINSERT INTO t3 VALUES(24999,442549,'four hundred forty two thousand five hundred forty nine');INSERT INTO t3 VALUES(25000,423958,'four hundred twenty three thousand nine hundred fifty eight');COMMIT;

SQLite 3.3.3 (sync):
1.778
SQLite 3.3.3 (nosync):
1.832
SQLite 2.8.17 (sync):
1.526
SQLite 2.8.17 (nosync):
1.364
PostgreSQL 8.1.2:
19.236
MySQL 5.0.18 (sync):
11.524
MySQL 5.0.18 (nosync):
12.427
FirebirdSQL 1.5.2:
6.351

Test 4: 100 SELECTs without an index

SELECT count(*), avg(b) FROM t2 WHERE b>=0 AND b<1000;select>=100 AND b<1100;select>=200 AND b<1200;...>=9700 AND b<10700;select>=9800 AND b<10800;select>=9900 AND b<10900;>Test 5: 100 SELECTs on a string comparison

SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%one%';SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%two%';SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%three%';... 94 lines omittedSELECT count(*), avg(b) FROM t2 WHERE c LIKE '%ninety eight%';SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%ninety nine%';SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%one hundred%';

SQLite 3.3.3 (sync):
4.853
SQLite 3.3.3 (nosync):
4.868
SQLite 2.8.17 (sync):
4.511
SQLite 2.8.17 (nosync):
4.500
PostgreSQL 8.1.2:
6.565
MySQL 5.0.18 (sync):
3.424
MySQL 5.0.18 (nosync):
2.090
FirebirdSQL 1.5.2:
5.803

Test 6: INNER JOIN without an index

SELECT t1.a FROM t1 INNER JOIN t2 ON t1.b=t2.b;
SQLite 3.3.3 (sync):
14.473
SQLite 3.3.3 (nosync):
14.445
SQLite 2.8.17 (sync):
47.776
SQLite 2.8.17 (nosync):
47.750
PostgreSQL 8.1.2:
0.176
MySQL 5.0.18 (sync):
3.421
MySQL 5.0.18 (nosync):
3.443
FirebirdSQL 1.5.2:
0.141

Test 7: Creating an index

CREATE INDEX i2a ON t2(a);CREATE INDEX i2b ON t2(b);
SQLite 3.3.3 (sync):
0.552
SQLite 3.3.3 (nosync):
0.526
SQLite 2.8.17 (sync):
0.650
SQLite 2.8.17 (nosync):
0.605
PostgreSQL 8.1.2:
0.276
MySQL 5.0.18 (sync):
1.159
MySQL 5.0.18 (nosync):
0.275
FirebirdSQL 1.5.2:
0.264

Test 8: 5000 SELECTs with an index

SELECT count(*), avg(b) FROM t2 WHERE b>=0 AND b<100;select>=100 AND b<200;select>=200 AND b<300;...>=499700 AND b<499800;select>=499800 AND b<499900;select>=499900 AND b<500000;>Test 9: 1000 UPDATEs without an index

BEGIN;UPDATE t1 SET b=b*2 WHERE a>=0 AND a<10;update b="b*2">=10 AND a<20;... b="b*2">=9980 AND a<9990;update b="b*2">=9990 AND a<10000;commit;>Test 10: 25000 UPDATEs with an index

BEGIN;UPDATE t2 SET b=271822 WHERE a=1;UPDATE t2 SET b=28304 WHERE a=2;... 24996 lines omittedUPDATE t2 SET b=442549 WHERE a=24999;UPDATE t2 SET b=423958 WHERE a=25000;COMMIT;

SQLite 3.3.3 (sync):
1.883
SQLite 3.3.3 (nosync):
1.894
SQLite 2.8.17 (sync):
1.994
SQLite 2.8.17 (nosync):
1.973
PostgreSQL 8.1.2:
23.933
MySQL 5.0.18 (sync):
16.348
MySQL 5.0.18 (nosync):
17.383
FirebirdSQL 1.5.2:
15.542

Test 9: 1000 UPDATEs without an index

BEGIN;UPDATE t1 SET b=b*2 WHERE a>=0 AND a<10;update b="b*2">=10 AND a<20;... b="b*2">=9980 AND a<9990;update b="b*2">=9990 AND a<10000;commit;>Test 10: 25000 UPDATEs with an index

BEGIN;UPDATE t2 SET b=271822 WHERE a=1;UPDATE t2 SET b=28304 WHERE a=2;... 24996 lines omittedUPDATE t2 SET b=442549 WHERE a=24999;UPDATE t2 SET b=423958 WHERE a=25000;COMMIT;

SQLite 3.3.3 (sync):
3.153
SQLite 3.3.3 (nosync):
3.088
SQLite 2.8.17 (sync):
3.993
SQLite 2.8.17 (nosync):
3.983
PostgreSQL 8.1.2:
5.740
MySQL 5.0.18 (sync):
2.718
MySQL 5.0.18 (nosync):
1.641
FirebirdSQL 1.5.2:
2.976


If you want to see some more information on the above and 10 more tests, you can go Here

Sunday, March 12, 2006

Flash 8 and Security

By Justin Silverton

I recently came across the following message when I tried to run a flash program on a client's machine:


















The security dialog comes up because when you fire getURL() with Local Playback Security set to
"Access Local Files only" it sees the getURL call as a request for network resource (and pops up the
security dialog).

If you then set Local Playback Security set to "Access Network".... Normally that would allow the
call access to the network. But the requested communication is actually between a local SWF and a
local HTML file, so it sees that as a local file accessing a local file, which is outside of what's
allowed when LPS is set to "Access Network". Which results in a Flash Player 8 Security Sandbox dialog .


For Developers, there are three ways to solve the above issue:

1. The end user has to use the Settings Manager to set local file security to "Always Allow" AND
they have to add the path to the file as a trusted path.
The direct path to this section of the online Settings Manager is
(http://www.macromedia.com/support/documentation/
en/flashplayer/help/s...).
The default is 'always ask'. Change that to 'Always Allow'.
Then add the path to your local content to the trusted locations. For example, if your content
is on a CD-ROM then you'd add the path to the CD (for example, "F:/").

Doing these two things is essentially enabling a local Trust File. Settings manager then
writes the trust file settings for you, to the #SharedObjects (which is obfuscated so nobody can
crack it)
So that's how you can do it if your users are internet-connected and you feel they're savvy
enough to handle the steps.

What if your users are not internet connected? In that case you have to manually add the trust file
to one of two locations:

2. You can create a trust file in C:\Documents and Settings\\Application
Data\Macromedia\Flash Player\#Security\FlashPlayerTrust.
The name of the file can be whatever you want.
The only minimum thing in the file is one line of text that's the path you want to trust.
Additional paths can be one per line.
Do this if you just want to set up trust for one unique user account on that machine.

3. You can create a trust file in C:\WINDOWS\system32\Macromed\Flash\FlashPlayerTrust.
This is the same trust file as step #2, but sets it for all the users on this machine.
The catch here is that you have to be an admin on the machine to create this trust file.

Options #2 and #3 are obviously also available to end users who do have internet connections but
whom you might not want to direct to the Flash Player Settings Manager.

Saturday, March 11, 2006

optimizing mysql tables

By Justin Silverton

Many times, slow access to a mysql database can be the result of Badly defined or non-existent indexes and fixing these can often lead to better performance. Here is an

example table:

CREATE TABLE address_book (

contact_number char(10) NOT NULL,
firstname varchar(40),
surname varchar(40),
address text,
telephone varchar(25)
);

example query: SELECT firstname FROM address_book WHERE contact_number = '12312';

This will retrieve the firstname of a person added to the address_book table, based on the contact number.

Without any kind of indexes added to this table, mysql will have to search through each row to find the item that you would like to find, which is very inefficient.

Optimizing your table

There is a built-in command called explain, that can show you what, if any, indexes that are being used to retrieve results.

example:

EXPLAIN SELECT firstname FROM address_book WHERE contact_number = '12312';

This will return a set of results that will tell you how myql is processing the results

table: The table the output is about (will show multiple if you have joins)
type: The type of join is being used.best to worst the types are: system, const, eq_ref, ref, range, index, all
possible_keys: Shows which possible indexes apply to this table
key: And which one is actually used
key_len: The length of the key used. The shorter that better.
ref: The column, or a constant, is used
rows: The number of rows mysql believes it must examine to get the data
extra: You don't want to see "using temporary" or "using filesort"

and index can be added to the above example table using the following command:

ALTER TABLE address_book ADD INDEX(contact_number);

you can also add an index on only part of a varchar. In the following, I will add an index on only 8 of the 10 characters.

ALTER TABLE address_book ADD INDEX(contact_number(8));

Why would you want to do this?

Indexes do increase performance in the right situations, but they are also a tradeoff between speed and space. The bigger an index is, the more space it will consume on your harddrive.

Using the query optimizer/analyzer

the following command can analyze your table key distribution to find out the best indexes to use:

analyze table *tablename*

also, another thing to keep in mind is the fact that over time, update and delete operations leave gaps in the table, which will cause un-needed overhead when reading data from your tables.

from time to time, it is a good idea to run the following (which will fix the above issue):

optimize table *tablename*

Thursday, March 09, 2006

What would you like to see in PHP 6?


By Justin Silverton

In November of 2005, the major developers working on the core of PHP met up in Paris and discussed various additions that would possibly make version 6.

The attendees


Marcus Börger (SOMABO)
Wez Furlong (OmniTI)
Rasmus Lerdorf (Yahoo!)
Derick Rethans (eZ systems)
Dmitry Stogov (Zend)
Zeev Suraski (Zend)
Jani Taskinen
Andrei Zmievski (Yahoo!)

The following are a list of the top 5 additions I think should be made (in no particular order):

1) Filename Encoding

issue: Files on a file system can have names encoded in different character sets. For example Windows can make use of it's UTF16 based filename API, while on Linux it simply depends on the application

2) optimizing []

issue: Currently using the [] operator to select an arbitrary character is very slow as PHP needs to start scanning the string from the start.

3) register_globals

issue: This causes many security issues and should be removed completly. I still encouter PHP scripts that make use of the this feature.

4) 64-bit integer

issue: With 64-bit processing in the horizen, this will come in handy for future (and some present) apps.

5) built-in opcode cache

issue: When an opcode caching system is installed (such as zend or ion cube), script execution time is much better.

A list of everything discussed can be found Here

Wednesday, March 08, 2006

patTemplate and php

by Justin Silverton

What is patTemplate?

patTemplate is a set of classes that allow you to separate your php code from design/output, making your code easier to maintain. To identify a certain part of the page as a template, patTemplate uses XML tags to assign a template a unique name and a various attributes (see patTemplate Tags and attributes for a list of all tags and attributes). When parsing a template, the parser divides the page in several chunks and treats them as separate templates. By using patTemplate's API you can hide, display or repeat a certain template.

Template Types

OddEven
you can assign to subtemplates () to this type: Odd and Even, which will be alternated, when the template is repeated

Condition
you can assign as many subtemplate as you like and you have to assign one variable which will be used to compare it with the specified conditions of the subtemplates. This allows you to define different HTML code for each value your condition variable may have. There are two special conditions you can use: default, which is similar to the default in a switch/case statement and empty which will be displayed, when there is no value assigned to the condition variable.

SimpleCondition
A situation that often arises is that an error message has to be displayed if an error occured. Normally you would set the error message if it exists or hide the template using setAttribute() if there is none. By using a template of type SimpleCondition you can assign a list of variables (using the requiredvars) that have to be set if the template has to be displayed. If one of these variables is not set, the template will not be visible.

Variables

patTemplate supports variables similar to programming languages. There can be local variables (assigned by addVar) and global variables (assigned by addGlobalVar).

Global variables can only be scalar variables (strings, integers,...) and local variables can also be arrays. If you assign an array to a variable, the template where you assigned the variable will be repeated.

If a condition template uses the useglobals="yes" attribute global variables will be used if no local variables are set.

Variables have to be written in uppercase and may only contain chars, numbers or the underscore (_). Variables are always enclosed in {}.

Variables will be replaced with their values when parsing a template.

There is one predefined variable you can use when you want to enumerate a template that is automatically repeated: just place the variable PAT_ROW_VAR in your template, where you'd like to enumeration to appear (can also be used for javascript mouseovers).

How to install

There are two ways to install phptemplate

1) through pear

Download the pear package and execute: " pear install patTemplate-3.0.1.tgz"

2) Manually

A) Download the .zip or .tgz manual package from the patTemplate website
B) Download the patError class

Example of usage


Example of usage

error_reporting( E_ALL );

//require the error reporting class
require_once( 'pat/patErrorManager.php' );
//require the main template class
require_once '../patTemplate.php';
//create a new instance
$tmpl = &new patTemplate();
//set the directory of where to find the templates
$tmpl->setRoot( 'templates' );
//use this file for your template
$tmpl->readTemplatesFromInput( 'example_api_addvar.tmpl' );
//add a variable for displaying
$tmpl->addVar( 'template1', 'argh', array( 'one', 'two', 'repeat it.' ) );
//actually display it
$tmpl->displayParsedTemplate();

Why use this template system?

There are many template systems available for PHP. I have recently started using this one over the others because it is simple, has less overhead, and is much easier for a person designing your pages to read and understand.

Download

patTemplate can be downloaded Here

Monday, March 06, 2006

5 reasons not to use oscommerce


By Justin Silverton

What is Open Commerce?

From oscommerce.com: "osCommerce is the leading Open Source online shop e-commerce solution that is available for free under the GNU General Public License. It features a rich set of out-of-the-box online shopping cart functionality that allows store owners to setup, run, and maintain their online stores with minimum effort and with no costs, license fees, or limitations involved.

The goal of the osCommerce project is to continually evolve by attracting a community that supports the ongoing development of the project at its core level and extensively through contributions to provide additional functionality to the already existing rich feature set."


Why Shouldn't I use it?

1) no separation of logic and presentation

Smaller applications can be created without separating logic and presentation, but when an application gets as large as oscommerce, there needs to be some kind of templating system in place.
A templating system can also be used to cache dynamic pages and improve the overall performance.

A good, scalable system needs to be engineered from the ground up. It looks to me like it was hacked together with pieces of code here and there

2) difficult to integrate into an existing design

out of the box, the cart works fairly well. If you want to make any drastic design changes, you will run into major issues.

Although it is free, and this may be intising to many companies, the time and labor cost of updating the cart to suit your needs ends up being more than many of the commerical carts available.

3) security

although it is updated, #1 makes it very difficult to make updates without having to manually open up each .php file and make the changes yourself.

4) cannot have multiple sizes of image previews (thumbnail, medium, large)

5) admin navigation issues

a) hard to do shipping cost per item (with different items having different costs) per country
b) editing product descriptions seems a little awkward. overall, it looks like it was developed for a programmer, rather than a store owner.

commercial

cubecart - http://www.cubecart.com/
sunshop - http://www.turnkeywebtools.com/
miva mercant - http://www.miva.com

open source

interchange - http://www.icdevgroup.com/

Sunday, March 05, 2006

using sqlite and php


by Justin Silverton

What is SQLite?

SQLite is a small library that implements a self-contained, embeddable, zero-configuration SQL database engine. Features include:
  • Transactions are atomic, consistent, isolated, and durable (ACID) even after system crashes and power failures.
  • Zero-configuration - no setup or administration needed.
  • Implements most of SQL92. (Features not supported)
  • A complete database is stored in a single disk file.
  • Database files can be freely shared between machines with different byte orders.
  • Supports databases up to 2 terabytes (241 bytes) in size.
  • Sizes of strings and BLOBs limited only by available memory.
  • Small code footprint: less than 250KiB fully configured or less than 150KiB with optional features omitted.
  • Faster than popular client/server database engines for most common operations.
  • Simple, easy to use API.
  • TCL bindings included. Bindings for many other languages available separately.
  • Well-commented source code with over 95% test coverage.
  • Self-contained: no external dependencies.
  • Sources are in the public domain. Use for any purpose.
Why would I need this?

If you want to have the advantages of a SQL database, without having to install a separate system such as mysql or postgres. It has also been demonstrated to work on websites getting more than 1,000,000 hits per day.

lots of good performance info can be found here
project page can be found here

Installing

Read the INSTALL file, which comes with the package. Or just use the PEAR installer with pear install sqlite. SQLite itself is already included, You do not need to install any additional software.

Windows users may download the DLL version of the SQLite extension here: (php_sqlite.dll).

In PHP 5, the SQLite extension and the engine itself are bundled and compiled by default. However, since PHP 5.1.0 you need to manually activate the extension in php.ini (because it is now bundled as shared). Moreover, since PHP 5.1.0 SQLite depends on PDO it must be enabled too, by adding the following lines to php.ini (in order):

On Linux or Unix operating systems, if you build PDO as a shared extension, you must build SQLite as a shared extension using the --with-sqlite=shared configure option.

SQLite 3 is supported through PDO SQLite.

Windows installation for unprivileged accounts: On Windows operating systems, unprivileged accounts don't have the TMP environment variable set by default. This will make sqlite create temporary files in the windows directory, which is not desirable. So, you should set the TMP environment variable for the web server or the user account the web server is running under. If Apache is your web server, you can accomplish this via a SetEnv directive in your httpd.conf file. For example:

SetEnv TMP c:/temp
If you are unable to establish this setting at the server level, you can implement the setting in your script:

putenv('TMP=C:/temp');
The setting must refer to a directory that the web server has permission to create files in and subsequently write to and delete the files it created. Otherwise, you may receive the following error message: malformed database schema - unable to open a temporary database file for storing temporary tables

Using SQlite

The following is a simple example that can get you started with SQLite:



<?php
// create new database
$db = sqlite_open("db.sqlite");

//create a table named test
// sqlite_query($db , "CREATE TABLE test (id INTEGER PRIMARY KEY, name CHAR(128))");

// insert sample data
sqlite_query($db, "INSERT INTO test (name) VALUES ('Imuvalue')");

// execute query
$result = sqlite_query($db, "SELECT * FROM test");
// iterate through the retrieved rows
while ($row = sqlite_fetch_array($result))
print_r($row);
// close database connection
sqlite_close($db);
?>

Friday, March 03, 2006

using php and postgres

By Justin Silverton

This is a simple guide on how to start connecting to a postgres database with php. I also assume that you already have a database setup with a table/tables.

Making a connection to the database

First, make sure the postgres module is enabled in php. This can done by looking through your php.ini file and ucommenting it in the modules section.

in windows, the line is: "extension=php_pgsql.dll"
in *nix, it is: extension="php_pgsql.so"

next, a statement needs to setup for the actual connect.

$my_connection = pg_connect("user=postgres dbname=phphh");

pg_connect() takes a single argument, a connection string, which consists of name=value pairs.

Other valid connection string names include:

host - an optional host name for the database
port - an optional port number for the database
password - a password for the user you are connecting as
options - options to pass to the backendtty - a UNIX terminal path to send debugging information to

Unless you have customised your PostgreSQL installation, you should not need to use these.

The pg_connect() function returns a database connection handle which should be used with the PostgreSQL utility functions. On error, it returns FALSE.

Queries

$query_resource = pg_exec($my_connection,"INSERT INTO mytable VALUES('value1','value2','value3');");

if(!$query_resource)
exit("could not insert into database");

$query_resource = pg_exec($my_connection,"SELECT * FROM mytable;");

if(!$query_resource)
exit("could not selectfrom database");

Other Helpful functions

This is a list of other helpful functions that can be used to get data.

pg_fetch_row() - retrieve a single row
setpg_fetch_array() - retrieve a single row as an array
pg_fetch_object() - retrieve a single row as an object
pg_result() - retrieve a single field
pg_numrows() - number of rows returned by the query
pg_numfields() - number of fields per row returned by query
pg_fieldnum() - the numeric number of a given field
pg_fieldname() - the name of a given field

Thursday, March 02, 2006

5 postgres tips

By Justin Silverton

Here are 5 postgres tips to follow my "5 mysql tips" post. These are known issues with versions 7.3 and below.

1) Object names that aren't in quotes are assumed to be lower-case

example: Select column1 FROM table1 and SELECT COLUMN1 FROM table1 both will return results from column1. SELECT "COLUMN1" FROM table1 will be viewed as a diffrent column by postgres.

2) count(*) is very slow with large tables

This is because rather than an indexed scan, postgres scans the entire table to determine the count.

3) Min and Max are both slow with large tables

This is fixed in version 8.0, but there is a work-around that I have used in previous versions:

min equivalent: SELECT column1 FROM table1 ORDER BY column1 DESC LIMIT 1
max equivalent: SELECT column1 FROM table1 ORDER BY column1 ASC LIMIT 1

4) unicode values

unicode defaults to UTF-8

5) integers used in calculations that result in a value that is greater that 32-bits will end up as 0.

to fix the issue, either get postgres version 8.0 or cast one of your values as a 64 bit value (select integer::INT8)

Wednesday, March 01, 2006

5 mysql tips

By Justin Silverton

These are some tips that may help you out when dealing with mysql tables (known in 4.1 and below).

1) char and varchar are case sensitive

example:

if you have a table that contains the following:

table newtable (
name varchar(32)
)

name contains the name "John Smith".

the following statement: "SELECT * from newtable where name='john smith' will return our record.

to stop this from happening, use the following when you create your table:

CREATE TABLE newtable (
name VARCHAR(32) BINARY
)

2) Varchar type is limited to 255 characters

3) Varchar trailing spaces are stripped

example: insert into newtable values('Test with no spaces ');

select concat(name, 'no spaces') FROM newtable;

output will be: Test with no spacesnospaces

Varchar works this way, because it saves space by stripping the spaces.

if you need to keep the trailing spaces in the data you are adding to a varchar type,
you need to use the text or blob types.

4) operator

The (or) operater is a logical operator

example: select 'string1' 'string2' will not return 'string1string2'

5) function parameters

This issue has caused me many headaches in the past, and I am not sure why this issue was never fixed. If there is a space
between the paramater list and an internal function that you want to execute, it will return an error.

example: select min (my_field) from mytable wil return an error, while select min(my_field) from my_table will not

How to manually remove a program in windows XP

By Justin Silverton

The following can help in those situations on windows, when you try and un-install a program and it either doesn't remove itself from the main add/remove programs menu or an error message comes up because the un-installer is corrupted.

Warning: editing your registry impropery can cause damage to windows. Use the following at your own risk:

1. Click Start, and then click Run.
2. In the Open box, type regedt32, and then click OK.
3. In Registry Editor, find the following key:

HKEY_LOCAL_MACHINESOFTWARE
MicrosoftWindowsCurrentVersion\Uninstall

4. In the left pane, click the Uninstall registry key, and then click Export on the File menu.
5. In the Export Registry File dialog box that appears, click Desktop in the Save in list, type uninstall in the File name box, and then click Save.
6. Each key listed under Uninstall in the left pane of Registry Editor represents a program that is displayed in the Currently installed programs list of the Add or Remove Programs tool. To determine which program that each key represents, click the key, and then view the following values in the details pane on the right:
DisplayName: The value data for the DisplayName key is the name that is listed in Add or Remove Programs.

-and-

UninstallString: The value data for the UninstallString key is the program that is used to uninstall the program.

7. After you identify the registry key that represents the program that you removed but which is still displayed in the Currently installed programs list of Add or Remove Programs, right-click the key in the left pane of the Registry Editor window, and then click Delete.

Click Yes in response to the "Are you sure you want to delete this key and all of its subkeys?" message.

8. On the File menu, click Exit to quit Registry Editor.

9. Click Start, click Control Panel, and then click Add or Remove Programs.

In the Currently installed programs list, verify that the program whose registry key you deleted is no longer listed.

10. Do one of the following:• If the program list is not correct in Add or Remove Programs, double-click the Uninstall.reg file that you saved to your desktop in step 5 to restore the original list of programs in the registry.

-or-

If the program list is correct in Add or Remove Programs, right-click the Uninstall.reg file on your desktop, and then click Delete.

Update

an anonymous poster has an addition to my post:

It's worth mentioning that this doesn't actually remove any of the files or reg keys the program created when it was installed, it just removes the 'record' Windows has which tells it the program has been installed.An alternate way to do this is to use msizap :http://msdn.microsoft.com/library/default.asp?url=/library/en us/msi/setup/msizap_exe.asp