All posts tagged 'projects'

DevOps?

After writing about Taco Bell Programming I discovered that there is a new movement called “DevOps” which goes into the same direction:

Let’s face it – where we are right now sucks. In the IT industry, or perhaps to be more specific, in the software industry, particularly in the web-enabled sphere, there’s a tacit assumption that projects will run late, and when they’re delivered (if they’re ever delivered), they will underperform, and not deliver well against investment. It’s a wonder any of us have a job at all!

The Devops movement is bold enough to believe that there’s a better way – a way of building teams, and building software that can solve these problems.

Read the full DevOps article.

Logwatch for PHP errors, the Apache error log and MySQL

Logwatch is a very flexible and customizable log watching system for lazy system administrators. It checks the logfiles regularly and sends custom mail reports – very useful and way better than daily manual checking of the logs.

Unfortunately the configuration for MySQL, PHP and the Apache error_log is missing; so let me share the configuration scripts:

Logwatch configuration for PHP
Interestingly there is no logwatch configuration for PHP error_log files – and all a search revealed are some outdated files which don’t work anymore. Here are the adopted files that allow you to get notified about PHP errors:

  • logfiles_php.conf – place this file in /etc/logwatch/conf/logfiles/php.conf and adopt the path to the php error_log file (or just use the Apache error_log)
  • services_php.conf – place this file in /etc/logwatch/conf/services/php.conf
  • scripts_php – place this file in /etc/logwatch/scripts/php and make it executable

Logwatch configuration for MySQL
Also a configuration for MySQL is missing in the default configuration; here are the configuration files:

  • logfiles_mysql.conf – place this file in /etc/logwatch/conf/logfiles/mysql.conf and adopt the path to the MySQL logfile
  • services_mysql.conf – place this file in /etc/logwatch/conf/services/mysql.conf
  • scripts_mysql – place this file in /etc/logwatch/scripts/mysql and make it executable

Logwatch configuration for Apache’s error_log
Now hat really made me wonder is the at the httpd access_log is monitored but the error_log is left out, so no details about errors are included in the logs. This configuration ignores PHP errors and includes all httd errors in the logwatch output:

  • logfiles_http-error.conf – place this file in /etc/logwatch/conf/logfiles/http-error.conf and adopt the path to the httpd error_log file accordingly
  • services_http-error.conf – place this file in /etc/logwatch/conf/services/http-error.conf
  • scripts_http-error – place this file in /etc/logwatch/scripts/http-error and make it executable

To test that the individual scripts work use this command:

/usr/sbin/logwatch –detail high –print –service $SERVICE –range today –debug 0

Replace $SERVICE with either php, mysql or http-error and set debug to 5 if you want to see more (but not too much) debugging information.

All the logwatch configuration scripts for MySQL, PHP and the Apache error_log can be found here.

Update: There was an error in the scripts_mysql file, fixed now.

Debugging using Apache as proxy

While the follow up article of “Must-have tools for HTML, JavaScript and AJAX development and debugging” (which has also been translated to Chinese, Japanese and Korean) is not written yet I want to share a useful trick that helps debugging live Web applications by injecting custom files into them.

The problem is how to test a new version of a JavaScript library on a production system without even touching the system itself in any way. Testing with local copies of the HTML pages work fine for a few files but does not scale very well, and staging systems might not be properly configured or really tell you if something works on the production system (bugs always just happen in production, right?).
Another problem with debugging JavaScript on production system is that the files might be compressed, which makes debugging in Firebug almost impossible – variable names are shortenend and line numbers don’t make sense any more – would be great to have the uncompressed version instead…

The solution is using a local proxy and replacing certain requests with local files. That way custom files can be injected based on regular expressions and tests can be performed on the production site or even on sites without access to the system itself.
While Fiddler works great on Windows and Charles Proxy does the same on OSX I want to present the poor man’s solution which makes use of any local Apache installation (i.e. MAMP).

The required changes in the httpd.conf are:

# Enable proxy requests
ProxyRequests On
ProxyVia On
<Proxy *>
# Secure the proxy to allow localhost requests only
Order deny,allow
Deny from all
Allow from 127.0.0.1
Allow from ::1

# replace some files with local copies
RewriteEngine On
RewriteRule myfile.js http://localhost:8888/test/myfile.source.js [P]

# Disable caching
ExpiresActive On
ExpiresDefault “now”
</Proxy>

Lets go through the changes line by line:
The first two lines enable the Apache Proxy support (ProxyRequests On); the <Proxy *> block then configures the proxy further. As stated in the documentation multiple times it is a good thing to secure the server, that’s why only access from localhost is permitted. The important part are now the RewriteRules – they proxy certain requests to the local server instead of the remote server, in the sample above all requests that contain myfile.js will be replaced by the version served from the local server. This could as well be any other remote machine serving the file, as the request is proxied again ([P] flag). Finally caching is avoided by setting all files to expire right now (while this adds more load to the proxy it is not that bad to do during testing).

Now the proxy has to be used in the browser or application of choice; once the proxy settings are changed within the networks settings the access log of Apache should become quite busy when browsing around and, depending on your mod_rewrite settings above, some requests should be replaced with the local copy of a file, making debugging way easier.

Of course the drawbacks of the poor man’s solution are that Apache has to be reloaded in case the RewriteRules change and changing the configuration is not as comfortable as with the dedicated proxy solutions mentioned above. Nevertheless this solution works, is fast and very reliable… and free :)

Practical Getting Things Done tricks

A while ago I started using GTD with my two tools of choice – Toodledo and Todo – and so far I’m very satisfied with the “results” and the progress. Time to recap my experiences with GTD, the tools and the process as it might help others, you, to get up to speed with Getting Things Done more quickly. Please note that these recommendations reflect my personal experience when adopting myself to Getting Things Done using modern tools, compared to paper files and folders as described in the de-facto standard book from David Allen.

It works!
Now that I used GTD for some time I can really say it works – my friends and peers sometimes might say “he is writing everything down – he can’t remember anything on his own”, but that’s not what should hinder you from using GTD. Writing everything down in a structured way really frees the brain from thinking about things it should not think about. Some of the lists I have are very long but I’m confident at any time that I know what I have to do so I don’t have to think about it anymore. Time you save and can spend on thinking about more interesting stuff.

Not everything should got to the “next actions” folder.
Per GTD definition almost everything should go through the Inbox – and in the first weeks I discovered that most stuff immediately after that goes to the next-actions list. Which does not make sense for two reasons:
1) If you next-actions list is very long you have to remember manually what tasks are more important than others, which wastes Brain-Processing-Unit (BPU) cycles.
2) If stuff remains in the next-actions list for a week or so it was no next-action in the first place.
After some time I became more confident that moving stuff to other categories also works fine and that I can also move items off next-actions as needed.

Do not review the inbox in the subway.
At least for me doing reviews of my list in the subway did not work out that well – most of my tasks are somewhat computer related – sending a mail, doing some quick research of a Web site, fixing a script or updating some text. These and a lot of other 2-minute items which require a computer do not work out in the subway. (Sorry Apple, but writing long mails on the iPhone is not what I want to do!)
Doing the review in front of a computer is way more efficient and faster, at least for me. And it is more efficient than to move those items from “in” to “next” before executing them…

There is also a medium priority.
In the beginning all my tasks had either no priority (90%) or a high priority (the remaining 10%) which does not really add any benefit regarding sorting. I see this problem as related to iTunes ratings – I don’t want to rate poor songs with a single star, as they don’t deserve it… so most songs remain unrated. Great songs on the other side get 5 stars, as they are great, aren’t they?
What I want to say is – find your way to use priorities – i.e. high if your life depends on it and it had to be done yesterday, medium for stuff that can is urgent, but not already overdue, low for important things and no priority for all other stuff. But use more than just high and none.

The context can save your ass (multiple times).
Use the context to store meta information about the tasks, i.e. “manager”, “girlfriend”, “call”… not for everything, but where it makes sense. That way you can easily access all items you want to discuss with your manager independent of the list they are in. And you can do that right now, as needed. Works great if your manager stops by and asks something – you can, within seconds, tell him all the new gadgets you need *right now* to survive your job! :)
Finally Appigo’s Todo also supports the context and tags it really makes sense to use that information as well!

Waiting-for is really useful.
The second-most used list is waiting-for – in there I drop stuff that I have delegated, where input from others or action of others is required. This is great as I cannot forget to follow up on actions or input – no more slipping of deadlines because there was no follow up on time. Also this list tells you what others are working on for you (or should be working on…).

Review the list regularly. The full list.
To have a working GTD system it is also important that you review your full list regularly. It is not sufficient to just review the next actions and the Inbox, you have to go through every single item at least once a week. This helps you to clean up the waiting for list and maybe some other actions have been resolved in the meantime. Also it helps you to be confident that your system works and you have everything “in there”.

Delete tasks if needed!
When you do the regular review also remove tasks – don’t let something stick around just because you added it a while ago. Feel free to remove things as needed. Especially if something is on the next actions list for quite some time – drop it or at least move it to any other list.

Is it actionable? If not – rewrite or drop it (or put it on a list).
I also learned the hard way that actions have to be actionable – which makes it sometimes hard but is a good chance to clean up your projects. On the other side there are some items which don’t make sense to be actionable – i.e. “buy milk” – I don’t want to write it that way, instead have a groceries and a shopping list, where you know it is a list with items that are similar important and that are no actions.

Once again: GTD works!
As I already stated on top – Getting Things Done works and will make you more productive and save you a lot of time. Getting started is not easy but there are good resources out there that help you (I can recommend this one).
Also it is important to find your way – there is no wrong or right way of doing it and also GTD is not a strict process but about giving you a general idea of how it might work – then go ahead and adopt the process as needed!

I hope these practical hints helped you implementing GTD on your electronic device of choice, iPhone preferred :)

jQuery on the Server

Last night I had an idea about a project I’m working on and how to make this extensible and flexible by using JavaScript code snippets which are to be added to the Java code. Sounds like a crazy idea but I remembered Mozilla Rhino which allows executing JavaScript from within Java. Issue 1 solved.

Also I’m a great fan of the jQuery library and would most probably like to use it on the server side to perform modifications – saves a lot of time in working with the DOM. Luckily I also remembered a post of John Resig which explains exactly that issue (ok, Google helped me to find it :) ) – executing jQuery inside of Rhino. Using an application server like Jaxer is no option as the server environment is given. Issue 2 solved.

So, downloading Rhino, downloading env.js, downloading jquery.js, putting the sample together and, guess what, nothing worked. Uff. Even simple selectors like jQuery(‘div’).length always returned 0. After a little investigation I figured out that up to jQuery version 1.2.1 it worked like a charm, but the newer versions did not work anymore. But why? Hard to figure if there are not error messages shown.

What to do? Firing of a note to John Resig and – surprise – he wrote back within a few minutes, pointing me to a new version of env.js which also works with newer versions of jQuery.

THANKS John!

So grab the new version of env.js and start using jQuery 1.2.6 on the server right now!