Perl is great but with so many dead links to what were booming sites before is sad. Perhaps a bad perception on my part but it seems to me that a lot of the open source projects seem to lack the wealth of community support they once had years ago.

No it is not like 1GB XML file, but more like 100 10MB XML Files, which would not be the problem if memory would not be leaked by PHP.

Some random thoughts;

Stream the XML – using something like a SAX API (not DOM which loads the entire document)

If you need to do some kind of calculation based on the entire document i.e. you need to see all the data to be able to produce the final result, you might consider storing intermediate values to file. At the far end of this way of thinking is Google’s MapReduce (http://en.wikipedia.org/wiki/MapReduce) – distribute the processing. The livejournal guys have something heading towards a Perl implementation of the same – see links here: http://del.icio.us/harryf/gearman . Another angle would be to consider an MMap based solution – in Perl check out http://cpan.uwinnipeg.ca/htdocs/Cache-FastMmap/Cache/FastMmap.html for example

One advantage of Perl is it’s attracted the type of developers that like to solve this kind of problem (unlike PHP). Browse CPAN – Perl’s XML tools a also good. And demand Unicode support (some like Perl have it… other languages don’t)

http://cpan.uwinnipeg.ca/htdocs/Cache-FastMmap/Cache/FastMmap.html

Using other language may mean you will have to duplicate certain portion of application functionality, leading to problems with maintainence.

But as soon as I started to write the first few lines of the deamon in PHP, I started to get some bad feelings about it (even if it would’ve fit for the task at hand and I was actually almost only using PHP at that time). I just realized that nobody was using PHP for anything that runs longer than a few seconds – and I just didn’t want to be the first one to hit the possible bugs.

I knew something would write about set_error_handler. Unfortunately it does not work.

Here is what documentation says:

The following error types cannot be handled with a user defined function: E_ERROR, E_PARSE, E_CORE_ERROR, E_CORE_WARNING, E_COMPILE_ERROR, E_COMPILE_WARNING, and most of E_STRICT raised in the file where set_error_handler() is called.

Here is little example with problem I’ve mentioned:

< ?php

function myErrorHandler($errno, $errstr, $errfile, $errline)
{
echo("Error !");
}

set_error_handler("myErrorHandler");

$a=false;

$a->method();
?>

Error handler does not help in this case.

Things I find inconvenient (possibly wrong is)

- Need to compile stuff. For scripts I prefer to be able to run them right after the changes.
- Class names. They are smart and standardized which does not make them pretty for my taste.
- Too much standardization. For example Connector/J has to be JDBC complaint which means it has to do many smart things which are required by specs.
- Love for complication. Java applications are typically designed “right” which makes them complicated. You can take a look at any stack trace posted for Connector/J bugs… it is rarely less than 20 levels.
- Too many third party extensions, many acre commercial.
- Not overly convenient to work with strings.
- Product of large company. With Perl and PHP it is relatively easy to reach developers.
- Fully OO. I like objects but not for 2 line scripts.

I guess most of them are just lame excuses the right reason would be I just do not feel like it. I enjoy playing with something which allows me to do things quick and dirty. Most of my tasks are far in complexity and reliability requirements from plane auto pilot.

Java is probably good for enterprise world but I do not expect it to get too much traction in Web words which wants applications to be quickly developed by students implementing prototypes for their ideas.