Michael's Barrel of Monkeys

I've got 99 problems

2011-12-19T13:50:00.001-08:00

I guest posted over at CloudBees http://blog.cloudbees.com/2011/12/ive-got-99-problems-but-scala-aint-one.html

Errors and Exceptions at OSDC

2011-11-20T15:02:00.000-08:00

I spoke at the Open Source Developers Conference (canberra, au) and did a talk on what turned out to be the erlang way of building apps:

Osdc 2011 michael_neale

View more presentations from Michael Neale

Google+ a place for interesting discussions

2011-10-30T01:58:00.001-07:00

Much as I don't need *another* social network, google plus has, well, some pluses ! Seems a good place for in-depth discussion - I expect that is largely due to the early adopter crowd, but we will see.

https://plus.google.com/u/0/109005609397751560350/posts/BtqhhEsek3J

Posted via email from Michael's posterous

Resilient processes (and erlang)

2011-10-16T18:20:00.000-07:00

So of late - Erlang has been a great tool I have been using as part of the day job.

Its headline feature seems to be concurrency - and whilst that is part of it - I think the real secret sauce is reliability. We have concurrent software, we have fast software, but we don't really have a lot of reliable software.

At its core it is quite simple: take the semantics of operating system processes, and their isolation from each other, and scale that down so you can have thousands of them. A typical Erlang application is made up of many processes (not to be confused with OS processes) - which are light, disposable and have their own memory heap, garbage collector etc...

I did a talk on this recently (and will be refining this for an upcoming talk):

Errors and handling them. YOW nights Sydney 2011

View more presentations from Michael Neale

It is worth noting that Erlang draws inspiration from both functional programming, and logic programming, both things near and dear to my heart, it is no surprising I found it quite pleasant in almost every way.

Elasticness and Clouds

2011-05-30T21:55:00.001-07:00

Amazon pretty much claimed the word "elastic" in computing when they delivered their Elastic Compute Cloud (EC2) years ago. One of the key features of this is, unsurprisingly, that it is elastic: you use an API (ideally) or a web interface, to provision resources on demand.

This works nicely - except when it doesn't. After years of experience with this - I have noticed that (in general) at the first sign of trouble the API will start failing, requests dropping, timing out. (sometimes that can even be an early warning sign of impending doom).

It is worth noting that most clouds seem to have similar issues regarding APIs - they often don't have the same quality of service that your servers get. This wouldn't normally be a problem, but a common strategy with infrastructure clouds is to make use of this elasticness (duh !) day to day for your operations, as well as recovery. Frustratingly, due to this behaviour you have to either accept this QoS limitation, or plan around it by consuming extra resources ahead of time. The latter approach somewhat undoes the benefit of having a highly elastic API - but here we are anyway.

Somewhere, there is a balance, but at the moment, the big users of the public clouds are treating them increasingly as a less than elastic resource pool (look up Netflix and their use of Amazon for an example of this). I can't help but wonder if this means APIs will fall out of favour for highly elastic workloads, or if the QoS of these APIs will improve over time...

Posted via email from Michael's posterous

How to waste a friday

2011-03-18T17:32:00.001-07:00

A unusually verbose writeup on network issues and what they will do at status.aws.amazon.com:

Repeated here for posterity (E-BS is a story for another day, but for now):

From 7:28pm PDT to 9:56pm PDT, a networking issue affected connectivity to a significant number of instances in the US-EAST-1 region. Affected instances experienced degraded network connectivity to the Internet and to instances in other availability zones.

The root cause of last night's issue was when a core network routing device experienced a partial failure. While the router was causing packet loss, the failure was not detected by surrounding network devices and therefore they did not automatically fail traffic over to redundant network paths as intended.

Additionally, our network monitoring tools failed to help our network operators locate the specific source of the connectivity issues. Once our networking team determined the location of the impact, they were able to identify the failing router and manually failed traffic routes away from it. At this point, all affected instances regained full connectivity.

We will be completely replacing the failed network device and our team will work on the failed device to understand the source of the failure. More importantly, we will be working to understanding why our network monitoring did not allow our team to quickly isolate the problem and force the manual failover to redundant network routes. We rely on this monitoring to help us deal with partial failures which defeat the normal redundancy built into high availability network architectures. We understand the impact this event had on some of our users, this just took us too long to figure out, and will be intensely focused on improving our monitoring and addressing the root cause of this failure.

Posted via email from Michael's posterous

CloudBees runtime

2011-01-31T23:48:00.001-08:00

So one of the things I have been working on a bit lately is now finally live:

http://www.cloudbees.com/run.cb

The short version: very much like GAE but without the google. On top of this, this works with the so called "dev@cloud" (which is hudson - no renamed to Jenkins - still with me?) so you can have a code push -> test -> run cycle all in one place (if you like).

Zero proprietary apis, the data remains yours of course. There will always be a free runtime in the cloud, I think it is necessary to "keep the dream alive" so to speak. Signup is free of course (you may need to put in your phone number so it can check you are human).

It has been nice building something that I want to use. It is a nice place to be.

Enjoy !

Posted via email from Michael's posterous

The Sun

2011-01-06T15:14:00.000-08:00

Thanks to generous subsidies (state government - it went way beyond what they expected and had to be wound down), a significant portion of my power is from the sun. Surprising how light the array panels are - and they have coped just fine through some furious storms.

Only a tiny system, 1.5k rating for the array, but it often gets close to 2K being generated on a sunny day (thankfully the inverter is over-rated for over 2K).

(photo above is panorama from the deck below the panels)

Homebrew: if you develop on OS-X...

2010-11-24T22:56:00.001-08:00

If you don't already have homebrew, run, don't walk, and install https://github.com/mxcl/homebrew

A really well done package manager for developers on OS-X (as opposed to a tool for administrators).

Posted via email from Michael's posterous

Getting running with Postgres (on OS-X)

2010-10-17T22:36:00.001-07:00

So I spent the best part of a friday morning "yak shaving" - getting postgres to run on OS-X in a development friendly way. I consulted The Google - and found thousands of ways one could install postgresql for development. I think the core of my problem is that a lot of the guides want you to have a hardened production installation.

So what I recommend (thanks to Simon Harris):

Install homebrew !

brew install postgresql

(have a coffee)

After it is installed, it spits out instructions on how to start, you may miss that, so here it is repeated:

If this is your first install, create a database with:

initdb /usr/local/var/postgres

If this is your first install, automatically load on login with:

cp /usr/local/Cellar/postgresql/9.0.1/org.postgresql.postgres.plist ~/Library/LaunchAgents

launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist

If this is an upgrade and you already have the org.postgresql.postgres.plist loaded:

launchctl unload -w ~/Library/LaunchAgents/org.postgresql.postgres.plist

cp /usr/local/Cellar/postgresql/9.0.1/org.postgresql.postgres.plist ~/Library/LaunchAgents

launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist

Or start manually with:

pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start

And stop with:

pg_ctl -D /usr/local/var/postgres stop -s -m fast

If you want to install the postgres gem, including ARCHFLAGS is recommended:

env ARCHFLAGS="-arch x86_64" gem install pg

Finally - this is all running as your current user - no super user, no postgres user. You can simply run:

createdb yourdbname-here

dropdb etc...

It will always use your current user - which is perfect for development. Much easier than trying to shoehorn in server grade configurations.

Another top tip, import a dump with all permissions stripped:

pg_restore --dbname=yourdevdb --no-privileges --no-owner yourdatabase.dump

Posted via email from Michael's posterous

Remote workers vs Distributed team

2010-09-14T01:15:00.001-07:00

This is one of those things that is obvious once someone clearly writes about it (something Bob is great at doing)

http://bob.mcwhirter.org/blog/2010/09/13/remote-worker-distributed-team/

Posted via email from Michael's posterous

The cloudbees blog

2010-08-29T17:20:00.001-07:00

I will shortly be blogging at the cloudbees blog - at least as soon as I can talk about what we are doing/what we have done without sounding like a complete idiot. Hopefully some useful lessons others can benefit from.

Posted via email from Michael's posterous

It's alive !

2010-08-26T18:48:00.001-07:00

So phase 1 of my new company is alive www.cloudbees.com or at least open to beta signups.

Its really "hudson as a service" with a few twists (which mostly make things faster and cheaper for users than they might initially realise).

Phase 2 is still being worked on... hopefully will have some more interesting things to say one day soon.

Posted via email from Michael's posterous

SJUG presentation on cloud APIs

2010-08-26T16:36:00.001-07:00

Sjug aug 2010_cloud

View more presentations from Michael Neale.

Ephemeral - the word of the week (cloud)

2010-07-13T17:34:00.001-07:00

So part of what I have been working on has been touching on 2 fairly popular "infrastructure clouds" Amazon EC2 and Rackspace CloudServers. Infrastructure as a service == IaaS == servers on demand (via some api, with illusion of infinite numbers of these resources available - lots more written elsewhere on the web so I won't repeat). I pronounce IaaS "yass" but a good friend of mine says "I Ass" which I may adapt, except in polite company.

In any case, there is a bit of a misunderstanding as to what "ephemeral" means for these virtual cloud machines: with amazons "instance storage" - people still seem to think that if you restart, you lose your storage, heck you even loose your ip. But this is NOT the case ! The only time you lose that storage is if the machine dies in some catastrophic way - or you terminate it. Restarting is just fine. In this sense Rackspace and EC2 are very similar (Rackspace has a higher QoS on their instance stores which is at least equaled by using an EBS store on ec2).

I am surprised that this confusion lives on, but reading the above paragraph for jargon its probably not surprising why. As always experience cures all this.

Posted via email from Michael's posterous

VMWare and owning the cloud

2010-06-17T17:57:00.001-07:00

I seem to be frequently asked about my thoughts on VMWare (and to some extent Spring Source) and their acquisitions, or rumoured acquisitions - in terms of what they are doing "with the cloud".

To me its pretty simple: Machine virtualisation is what enabled the so-called cloud computing evolution to start happening - making use of excess physical capacity, portable workloads, management apis etc. - and continues to be the cornerstone of clouds (with the notable exception of newservers.com !).

Of course, my personal first experience with virtualisation was with VMWare itself - usually in places where it was used to slice up machines for multiple test environments. I would bet it is the same for many. I think VMWare see themselves are the originator of this revolution/evolution and see it as their rightful place to be a top tier cloud service provider (both in terms of software, and providing, possibly, publicly available servers on demand). It might mess with their software licence business model, but to them, it doesn't matter, as they see themselves has having a right to be up there with ec2, with rackspace etc ! Good luck to them.

Posted via email from Michael's posterous

Awesome video/talk on personal motivation

2010-06-17T17:51:00.001-07:00

Posted via email from Michael's posterous

If getting errors in an IDEA scala project...

2010-06-02T20:01:00.001-07:00

Just a quick note, if you are getting lots of errors when trying to build a scala project in IDEA (typically one you just created) try this:

Open the module settings, click on the scala facet (if you don't have a scala facet then you need to add one !) - then uncheck the option that says "U

se scala library from specified jars" - instead, ad the appropriate scala-compiler.jar to your modules dependency list (but don't export it) - and away you go ! It has been this way for a while, never found a suitable explanation why, but hope this helps others if they see this:

Posted via email from Michael's posterous

SJUG March 2010 Restful design

2010-04-08T20:54:00.001-07:00

My recent presentation at the Sydney JUG on restful design (went pretty well !)

SJUG March 2010 Restful design

View more presentations from Michael Neale.

Speaking at SJUG - April 09

2010-04-05T18:50:00.001-07:00

I am speaking at the Sydney JUG this thursday - on RESTful design principles and practice. It is at the Atlassian offices in the city (corner Sussex & Market Streets - go around the back).

Posted via email from Michael's posterous

Leaving...

2010-03-30T17:17:00.001-07:00

Well it has been a blast - but after some time, I am leaving Red Hat - when I can I will blog more on the details of what is next for me.

I had a great time there, was privileged to work with people way smarter than me, on projects that matter to the world of open source, and are widely used. I got paid to work on open source. I got paid to work with ruby, scala, python and a whole heap of "cutting edge" things as well. I learned a lot. I can't really imagine there could be a better place to work. I got to work when and where I wanted, often on what I wanted. Wait - what the hell am I doing again? I must be completely insane.

So long.. and thanks for all the fish.

Posted via email from Michael's posterous

Chromium updater for mac

2010-02-28T00:23:00.001-08:00

Click here to download:

ChromiumUpdater.zip (86 KB)

A simple utility to grab the latest nightly of Chromium for OS-X and run it (see attached).

Just run this when you want to start chromium, it will get the latest nightly build.

Posted via email from Michael's posterous

The forgotten service: DNS

2010-02-14T21:52:00.001-08:00

I have been guest blogging over here - for example http://cloudpress.org/2010/02/09/domain-name-system-the-forgotten-cloud-service/

Posted via email from Michael's posterous

A new project

2010-02-05T20:09:00.001-08:00

I have had a need to have a simple DNS server I can run (for testing "cloud" related services) - and I came across EagleDNS - a authoritative DNS written in java (by Robert Olofsson). I was able to patch it to automatically reload zone files (it uses the same zone file format that BIND and others use, and are referred to in the RFC) when they are changed - something that seems strangely challenging for the more common DNS servers.

Happily my patch has been accepted and I am now a contributor: http://www.unlogic.se/core/projects/eagledns/page/changelog-1.0.1

I have spent way too much time looking at and thinking about and understanding DNS - but still, I guess learning about the fabric of the web is character building. Or something.

Posted via email from Michael's posterous

Simulating push messaging - the long poll

2010-01-17T14:35:00.001-08:00

I often end up in discussions on how "push" messaging works to a browser, a phone or a client behind a firewall/NAT. The use case being that the device appears to get real time notification that something happens, a message pops up etc... without having to poll for it.

Really, they all use the same mechanism, just over various protocols. In all these environments, you can assume that the device can not be reached from an "inbound" connection (ie a network connection that starts outside, and wants to reach the device) - but only in response to the device reaching out over the network to a server.

So the solution is to have a "long poll" or "slow poll" - in http sometimes it is called a "hanging get" - I think this is also similar to what HTML5 is standardising as WebSockets (but I haven't looked specifically at it). The client device opens a connection (http or whatever) and the server end just lets it hang - holds it open (via whatever means) - and either it times out (in which case the client just reconnects, and it starts again) or the server has a message to send to the client, which it then sends as the reply, immediately. Thus you get something that is close to push in terms of the user experience (the only time there would be a lag would be between a timeout and a reconnect - but that would be milliseconds, ideally, so not too noticeable).

It really isn't a huge mystery, and has been done for a long time. Unfortunately, on my iphone at least, it just ends up chomping batteries, but that is another story.

Posted via email from Michael's posterous

More people using scala?

2010-01-17T01:32:00.001-08:00

In my presentation at OSDC, a few big users of scala were mentioned, I thought it would be interesting to add EtherPad (which was aquired by Google - their team members are now on the Google Wave team - and the EtherPad codebase was open-sourced as part of this) and my new favourite toy: Foursquare (location based social app).

Posted via email from Michael's posterous

Some recent presentations

2010-01-16T23:53:00.000-08:00

The end of 2009 was fair busy travel wise for me (a mixture of holiday travel and work travel - Singapore, Byron Bay, Brisbane and Coffs Harbour) and I forgot to post some presentations I did recently that were pretty well received.

I attended the Open Source Developers Conference in Brisbane in late November, and did a couple of presentations.

On my own I presented on Complex Event Processing (best left the explanation to the slides):

Osdc Complex Event Processing

View more presentations from Michael Neale.

I also did a (now mandatory) scala presentation with my friend Tom Lee (who I often pepper with questions of an arcane nature about scala or python). We focussed on the good and easy and fun bits of scala, and how it compares to dynamically typed languages.

On Scala Slides - OSDC 2009

View more presentations from Michael Neale.

Both presentations went down quite well, were well attended, and some great discussion happened. The conference was a bit quieter than last year in Sydney, but had a nicely chilled vibe (probably due to being in the bush in Bardon, just west of Brisbane city).

Launching applications in OS-X "minified"

2009-12-03T16:12:00.001-08:00

Santa recently got a hackintosh for a certain nice little girl. To fit
photobooth on the screen, he had to use an (apple) script like:

do shell script "defaults write -g AppleDisplayScaleFactor 0.8"
tell application "Photo Booth" to launch
do shell script "defaults write -g AppleDisplayScaleFactor 1"

Handy ! launches the app at 80% size (need to reset the scale after it
is launched).

Posted via email from Michael's posterous

From the Future, another angle on concurrency

2009-10-26T03:37:00.001-07:00

(Warning contains graphic abuse of the type system in Scala code).

In languages like Scala, and of course Erlang, a lot of the focus is on Actors and message passing as a way to achieve concurrency and parallelism without using share-memory threads.

This is great, works awesome, and I love it. But its not all of the story. Sometimes you have a need to "fan out" a whole lot of slow/blocking things, and then join it all back together later on - a classic example used is hitting up a lot of remote resources at once (and very easy to do). Or, perhaps you have quite a few processors available, and a few computationally heavy tasks to do etc...

Well despite the bashing multi-threaded programming gets, the reality is threads are here to stay. But they are hard and I can't really do then.

Thankfully there are those smarter than me that have built lovely frameworks that take away most of the management of threads from us mortals (such as Doug Lea and java.util.concurrent on the JVM, and Grand Central Dispatch on OS-X - (and soon to be in BSD !)).

Speaking to Jon Tirsen of Google, he was musing over the idea of a language based around 100% async dispatch where everything was returned as a Future - pretty much the opposite to to lazy languages:

So I looked into java.util.concurrent.Executors - you can create executors (based on thread pools that you don't manage) and pass it "callable" tasks to run asynchronously. The trick is getting results back - this is where Futures come in. So when you submit a task to an executor, it returns a Future immediately - the future is a promise of a value at some point. When you call "get()" on the Future, it then blocks until it has a result (or course, hopefully it already has one). In java, this is understandable cumbersome, but works:

ExecutorService service = ...

Future result = service.submit(new Callable() {

public String call() {

return ... expensive computation ...

}

});

So result is of type Future of T - where T is the type of the real result. You have to call result.get() and then it will block... ugh, crazy... but worth it.

Enter Scala, before creating a whole new language as Jon suggested, I thought I would give ordinary Scala a go:

Wouldn't it be nice to take a piece of code like:

val result = someExpensiveFunction(42)

and make it async, and not worry about Future and all that? Well by the power of greyskull, thunking and implicit conversions this is possible:

val result = future { someExpensiveFunction(42) }

Note that I didn't have to change anything other then wrap future {.. } around it. Now, as Scala has type inference, this looks cleaner, but future is of type Future[T], where T is the return type of someExpensiveFunction. However, due to implicit conversions, we can pretend it isn't - it will return immediately, and you can pass it around, and once it actually needs the value of result, then, and only then will it block (automagically) - hopefully by then it will have its result !

So, how does it work:

object FutureProcess {

val executor = Executors.newCachedThreadPool

implicit def fromFuture[T](future: Future[T]) : T = future.get

def future[T](lambda: => T) : Future[T] =

executor.submit(new Callable[T] { def call=lambda })

}

So the above is all of it.

we create an executor (its in an object so we only have one instance of it, easy). In this case its just a thread pool (but you could size it based on CPU cores if you like).
we have an implicit def, which when it sees a Future of T, but needs T, it will call "get" (ie block and wait for result).
the Future function: take any "block" of code, which returns T, and submit it wrapped in a Callable (nice how compact it is). The only clever bit is the "thunk" - the "=> T" bit - which means it is a parameter which isn't evaluated when it is called, but when it is needed - this is important as it means it isn't evaluated until it is safely submitted to the executor (and thus has a separate thread doing it) and can return immediately.

So the happy result is that all you do is import FutureProcess, and then wrap future { ... } around the bits you want done async, but really, no other code changes. Classic scala, slightly mind bending at the library level, but a breeze to use.

Enjoy.

New podcast episode is up

2009-10-07T16:06:00.001-07:00

http://asylum.jboss.org/

Still getting the hang of mixing it - more compression needed. And
cowbell. 0% cowbell in this episode, unfortunately.

Posted via email from Michael's posterous

A visual comparison of a java to scala port

2009-09-28T23:26:00.001-07:00

... of a single class. In this case port was mechanical, I only did to
avoid mixed java/scala source in the one project (which you can of
course do, but I don't like doing it, and it was just one class).

I didn't change the code really (except for dealing with a null, which
is just too nice with a match).

You can guess which one is which (the scala one still ends up a little
bit shorter, mostly due to the constructor magic) but a whole lot more
pleasing to my eye anyway.

See and download the full gallery on posterous

Posted via email from Michael's posterous

A new podcast

2009-09-24T05:49:00.001-07:00

I recently started working on a podcast on all things jboss.org, R&D
related, with Max Andersen and Emmanuel Bernard. I am the only one
without an accent ;)

Editing podcasts is much less fiddly then editing music, but still
takes work and practice.

Content is syndicated through libsyn here: http://asylum.libsyn.com/

It is also available on itunes of course.

Posted via email from Michael's posterous

Deltacloud.org...

2009-09-23T21:35:00.001-07:00

I recently took a look at Delta Cloud - a good start at providing some
ability to switch "between clouds" or at least work across them
(including "clouds" based on virtualisation managers like RHEV-M from
Red Hat and VMWare).

Rackspace seems pretty sweet, has a nice RESTful style API to manage
stuff, so it was easy to write a driver for deltacloud.org (which I
did):

Get the good stuff here: (should be upstream shortly)
http://github.com/michaelneale/driver-rackspace/

The deltacloud framework itself is a RESTful rails app (and the
"drivers" are written in ruby).

Posted via email from Michael's posterous

Infinispan - distributed cache/data grid

2009-09-16T16:05:00.000-07:00

One has to have a hobby. Tragically mine is perilously close to work. So as part of learning about Infinispan (a distributed data grid) I created a RESTful server front end for it (in scala, of course), posted about it here.

Book review: Dependency Injection

2009-09-15T16:46:00.000-07:00

My good friend Dhanji Prasanna from Google recently finished his book on DI practices and patterns. I was given a copy for review - I am by no means a DI expert (barely know what it is).

To be honest I am/was a DI skeptic, I view it as a way to make up for weaknesses of statically typed mainstream languages (such as C# or Java). And thus DI brings in a raft of type unsafety issues, leading to runtime hassles (without the productivity/joy of a dynamically typed or scripting language).

However, the book always circles back to how to solve a problem in a type safe way - were code is more correct (and the compiler verifies it) before any runtime error occurs (any runtime DI activity is for performance and functional) - and leads to less surprises and biting in the ass.

The style of the book seems to be to show the problem in its raw form, using various tricks to try and get around it, failing and then introducing a DI framework, and showing the various solutions with the framework. Each section usually resolves to a simple solution that provides compile time type safety - I can reasonably assume this is Dhanji's intent, to gently "show you the way" - ie "the right way" of doing it. Dhanji is a Guice committer, so of course the best side of Guice is shown, but I believe he also does the best with the other frameworks he has a lot of experience with (personally I admire Guice a lot, I think it has learned from all the frameworks before it and been battle hardened).

Well if you do large scale java apps, you probably want to have someone on the team have this book.

Quick update - a Scala presentation

2009-09-15T00:07:00.001-07:00

I did a fairly well taken presentation at the Sydney JUG recently (trying to convince scala skeptics), thought I would put it here:

Scala Sjug 09

View more documents from Michael Neale.

Errata: I made a few boo-boos in explaining things:

I didn't realise you could "return" at any point in a function, but you have to explicitly annotate the return type of the function, for one ;)
The Eclipse plugin seems to be useful to some people (wasn't great last time I tried).
In ScalaCheck, you have to do a little more work to make it generate Arbitrary Instances of non "built in" types.
Probably many more...

Web apps are cheaper then desktop apps

2009-08-17T02:55:00.001-07:00

I find it interesting that Google, in all their might, still consider the reasons for targeting web apps to be (at least in part) economical:

http://blogs.ft.com/techblog/2009/07/app-stores-are-not-the-future-says-google/

I mean, if even Google balks at the cost of multiple client platforms (mobile and otherwise) what hope to mortals have?

Thus for many things web apps (as opposed to web sites) are the future. I am sure there is a kernel of apps that require desktop presence for the time being (video and audio editing, development for example) but otherwise, it looks like we have to look towards web tech for the future of client side computing.

JAOO - Sydney and Brisbane

2009-05-17T18:16:00.000-07:00

I was fortunate enough to speak at the JAOO conferences in Sydney and Brisbane recently, my talk is below (thanks to slideshare).

JAOO is an impressive conference, with a good (and no doubt expensive) list of international speakers. I highly recommend it, as a "bring the world to you" local conference (next year will be Brisbane and Melbourne I hear) - although the full price is quite an ask in this day and age....

Well my thoughts on interesting sessions I went to:

How to build an iPhone app in 45 Mins (Patrick Linksey):
Lots of fun - probably most popular session - a bit of an Objective C intro to those that didn't know it. I was quite enjoying the nuances of it until I realised the bits that were entertaining me was the manual management - I guess after almost 10 years of not worrying about it you forget the bad...
Speeding Ducks (Avi Bryant):
Great talk on VM performance - the sydney one turned into a bit of discussion/hand wringing on who would pay for the development of advanced VMs for Ruby... interesting...

Consistency, Storage, and Reliability in the Cloud (Jonas S Karlsson):
Jonas introduced Google's Megastore, which introduces some interesting bits from RDBMS into googles big table storage system - including transactions etc... (its not "ACID" but it does allow things to scale in the way that google needs them to).

Deception and Estimation: How We Fool Ourselves (Linda Rising):
This was more a shallow dive into psychology, why we willfully deceive ourselves, and why its probably a good thing... very interesting, and actually a lot of fun to listen to (I got the impression people felt relieved and uplifted by it).

Of course Josh Blocks puzzlers was fun: note to self, don't ever ever use inheritance again if you can at all help it (all the questions are secret as he is doing the presentation I think at JavaOne).

Writing Large Applications in JavaScript (Douglas Crockford):
Great talk on javascript in general, the the hilarious way it inserts semicolons (a dirty hack even by my low standards !).

A picture from the stunning conference centre in Sydney:

Jaoo Michael Neale 09

View more presentations from Michael Neale.

Single vendor hell - aka "one neck to choke"

2009-03-25T14:46:00.000-07:00

A colleague lamented the other day about the uber consolidation that is going on with the largest of IT vendors (and has been for some time).

I can't help but think its the "IT consumers" who share part of the blame for this: For years, in the late 90's and early 2000's people wanted to have less vendors to deal with, less overlap and "one neck to choke", well, guess what:

It has kind of happened. But this is a terrible terrible thing for IT consumers: who on earth would think that it could work out well for "IT consumers" to have one vendor providing the full stack? Its a testament to the genius of the vendors that the convinced people that a single vendor providing all parts of "the stack" was a good idea and a worthy goal.

Choice is a tiring thing to have, but its also a good thing, like exercise.

Be careful what you wish for - it might just come true. So IT consumers will now need to be doubly diligent to have a few suppliers in the mix lest the obvious happen to them.

Back it up

2009-02-09T15:04:00.000-08:00

Backing up personal data is like flossing, you know you should do it...

Personally I use time machine on the macs in the house, on other boxen (linux) I tend to copy/rsync home directories. But in many cases most data I care about is "in the cloud" (so I have washed my hands of responsibility of its safety - now is that a good thing ???).

What does interest me though is all the many ways that specific server apps demand they be backed up. Isn't that annoying for operations people? They need to have DBA/specialists for every single app to know how to back it up, each has their own process.

This is silly and not sustainable, thankfully modern filesystems can help. When you think about it, it is obvious. A backup really needs to help you out with 2 things: 1) allow you to recover in the event of a disaster of some sort - so you go back as recently as possible and continue processing. and 2) go back to a point in time.

File system snapshotting (via ZFS, or using virtual machines, whatever) seems good enough for most of this (#2 can be a bit of a problem - you need to keep sensible snapshots frozen in time, and be vigilant to watch out for data corruption inside an app - having versions of data inside an app can help this, but this is then not an IT infrastructure issue, no generic solution can help).

Certainly a whole lot less hassle for IT if they can just reset elements of infrastructure back to how they were at a single point in time. Everyone wins.

Poor mans lambdas

2009-02-04T16:38:00.000-08:00

A lot of my recent work has been in scala (I try to do any "green fields" work now in scala) so closures/lambdas are very pretty and flow freely in scala.

I did however read with interest about "lambda4jdt" a way of making code in java "look like" it is using a real lambda, but only really as a UI trick in eclipse. Interestingly I recall Cedric Beust talking about this previously (that is, the idea that IDEs can change the appearance of code from the underlying character stream).

Lambda 4JDT:

So screw the dead JLS (Java Language Spec), just go an add it in as low cal eye candy ! I like it !
With some excitement I went to IntelliJ's bug tracker to put in a similar feature request (I always struggled to explain what sort of "code folding" I wanted). I found out in a matter of minutes the feature is coming in IDEA 9 ! Nice !

In early versions, they used an actual lambda character:

(apparently it won't be exactly like that in the final version).

In any case I prefer scala in almost all ways, but for existing/legacy code, nice to have smart IDEs to present things as they should be.

MySQL developers denied .au visas due to local competition

2009-01-12T14:13:00.001-08:00

This is madness: http://blogs.mysql.com/kaj/2009/01/12/on-open-source-and-open-competition-in-a-not-so-open-world

Very disappointing.

Reading old code

2009-01-05T19:00:00.000-08:00

I am mostly terrible at reading old code, ESPECIALLY my own. I sort of stare and curse at it for a while, eventually it makes sense. Normally having runnable unit tests make it much easier.

However, I found some old ruby code of mine (a pre-historic simple rule engine, using YAML as the syntax - called "ruby rules"), and strangely it made sense (and I swear I have no recollection of writing it):

- rule: Foo
 set: Driver, Vehicle
 if: driver.age < 21 and vehicle.high_performance
 then: reject application



And a programmatic rule:


#now lets  try it out
rule = Rule.new :declarations => ["a", "b"],
               :types => [String.class, String.class],
               :condition =>"a == '42' and b == '42'",
               :action => "puts 'PASSED RULE 1'"

So what does this mean? I guess people smarter then me are right - ruby is a really friendly language. At least following the popular conventions for it leads to code that you could read well into the future (even without the comfort of static type guarantees).

OSDC slides/presentation details

2008-12-05T18:28:00.000-08:00

I just got back from OSDC and my slides are up on slideshare.

3 year anniversary

2008-11-30T15:03:00.000-08:00

It was 3 years ago (way back in innocent 2005) that I joined JBoss (as it was then). Since that time quite a bit has happened - I am grateful to have traveled over lots of the world, met great people and had a whole lot of fun.

JavaScript and the Next Big Language - problems

2008-11-25T15:05:00.000-08:00

Steve Yegge is famous for talking about Javascript as the Next Big Language.

I believe he was talking about Javascript 2/ECMA Script 4, which is currently in a "holding pattern". I personally believe this is great for the browser implementers and the web, but terrible for javascript growing into a language suitable for programming in the large (which is what I was hoping for some time this year).

Joel S mentioned once that he views C++ as a failure/disappointment (of course being a failure doesn't stop him and millions of others from using it) because it failued to have a String "type". In his words: "String is too important to be left to library implementers".

I would like to generalise this concept: while many popular languages can quite happily be "extended" by libraries, this doesn't mean that the language designers should leave out important features and leave it up to library writers to provide implementations just becuase they can (as it almost always turns into a zoo in about 30 seconds).

Some features are just too important to be left to library writers ;)

The Real web apps stand up

2008-10-29T19:23:00.000-07:00

There are 2 types of people in the world. Those that build user interfaces, and those that have an opinion on how they should be built (and unfortunately try to build frameworks).

The term "web app" came to mean apps that you access through a web browser. Most of them are (still) not a whole lot more then 3270 screen apps with colour. Actually 3270 devices were slightly smarter then most web apps. Even today, many web apps (and their supporting frameworks) follow this model. Since about 2000 people started using and abusing javascript to give these apps some of the responsiveness that we had back in the 70s. Yay. The term Ajax was coined when people realised they could fetch data in the background for these elements.

GMail came along and showed the first "mainstream" Real web app. It is just one "page" - which then uses the browser as the powerful GUI platform it is to render itself, in the client, and handle user gestures, in the client. What a client was meant to be, what "MVC" was meant to be. Kind of a simplified version of what people did 1000's of years ago with those xerox smalltalk and lisp machines.

This is not to say that the "ye olde" style web apps don't have a use - they certainly do. In fact the web is and should be mostly of this type of app/site (which is really a web site with dynamic content behind it). But they aren't really Applications from the user experience - at least not all they could be. Wonderful frameworks like rails seem to take things as far as they can go for content driven web sites - but most people find they want more (more importantly their users expect more) for richer apps (this is only a small subset of app out there). So you either have to find a front end Real app framework for your big complex app (like google and others do), or don't let your apps get complex, build smaller apps that can live as content driven web sites (like 37 signals do) - both are great approaches. Its just upsetting when people get confused (and you get a web site trying to be a big ugly complex app, or a hulking app for something simple that could have been done as a web site).

Some popular examples:

GMail - an app
Facebook - a web site with a little ajax (increasingly becoming an app - but it still relies on page refreshes)
Twitter - a web site (and rightly so, twitter is perfect as it is).

Well, no one could say it better then the authors of the truly amazing Cappuccino framework:

Designed for Applications

Nobody will deny that there is a distinct difference between a web site and a desktop application. Similarly, we believe there is a big difference between a static web page and a full fledged web application. Cappuccino is designed for applications, not web pages.

Instead of doing all or most of the work on the server, Cappuccino applications do as much as possible in the client. A typical application would never reload, but rather send and recieve data using traditional AJAX techniques and then present that data in the client code. 280 Slides is the first Cappuccino application, and it's a showcase of what is possible with this new framework.

Instead of worrying about how to implement drag and drop, copy and paste (of text and objects), undo and redo, document saving, rich cross-browser drawing and graphics, and a slew of other features, developers are free to focus on specific problems like PowerPoint support, or Twitter integration, or whatever else makes their application unique and compelling.

How does Cappucino Compare to Other Frameworks?

Cappuccino is not designed for building web sites, or making existing sites more "dynamic". We think these goals are too far removed from those of application development to be served well by a single framework. Projects like Prototype and jQuery are excellent at those tasks, but they are forced by their nature to make compromises which render them ineffective at application development.

On the other end of the existing frameworks are technologies like SproutCore. While SproutCore set out with similar goals to Cappuccino, it takes a distincly different approach. It still relies on HTML, CSS, JavaScript, Prototype, and an entirely new and unique set of APIs. It also requires special development software and a cumbersome compilation step. We think this is the wrong approach.

Bob on Rails

2008-10-23T15:27:00.000-07:00

My good friend Bob McW (Codehaus.org founder, who actually started Drools waaay back, and who is responsible for the name "Drools), is now back working in JBoss/Red Hat in Research and Prototyping.

He is working on making Rails deployable "natively" (in place rails apps, using all the usual rails infrastructure and deployment stuff) to finely tuned JBoss AS (5). As a bonus he is making clustering easy, and demoing the whole lot on "The Cloud".

It was some time ago that I noodled with getting Rails to run on JBoss on Fedora, but its nice to see bob taking this very seriously.

Cross platform icons

2008-10-20T19:24:00.000-07:00

And by cross platform icons I don't mean some format that caters to all the whims of gnome, windows, or OSX etc.. but I mean icons that don't look too out of place on each platform.

Eclipse !

These can work quite well in a web app. Of course I have made the mistake in the past (well I still do) of only providing icons - the "hieroglyphic" user interface it is sometimes called. Whoops. Well they are nice complement to text, buttons, menus etc, if used sparingly can draw the eye to what matters, and provide a little consistency. Plus they don't look too out of place on different operating systems.

The eclipse people have done a great job !

Its all here in a zip.

Probably the last blog post (hopefully) before I take some time off for "paternity" leave for child number 2.

GWT and closures talk

2008-08-17T22:38:00.000-07:00

I spoke the other day at the Sydney Java Users Group on my experiences with GWT, and I then attempted the dodgiest segway into the closures debate, with some examples in a variety of JVM languages (Scala, scheme, jython etc).

Anyway, here are the slides. I believe it was video'd too, as the slides won't make a whole lot of sense without the context (I did a lot of live coding).

My opinion on the closures debate depends on what mood I am in, and how much I am prepared to argue with people (in general I think the Java language needs a LOT of work, and it should happen, breaking backwards source capability if necessary).

Following is the view off the back deck of my new office (the office itself has a deck, which is nice and sunny):

Why is usability and user experience terrible in open source apps?

2008-08-09T20:16:00.000-07:00

I am a big fan of the Drunk and Retired podcast. Recently they had some discussion on why proprietary software, in general, has a better user experience then equivalent open source software, which is a fair generalisation.

I think Charles hit the nail on the head, its really painful work. Sometimes its quite hard, most times its just really really painful.

And open source is generally built either by volunteers (as I do part time) or paid staff (as I do full time) - but generally even in the case of the latter it is with a fraction of the level of investment that commercial vendors bring to bear on equivalent products, hence user experience can tend to suffer for a bit (certainly its probably the last bit that gets looked at).

Also - as different areas of software become “commoditised” its a great way for pay-for products to differentiate themselves - invest a lot in usability and it makes your product more likely to sell itself.

But I think its also a bit of a lost art - for some reason its seen as unsexy to care about user interface, or user experience compared with “hard core” parts (eg linux kernel developers are revered as having some magical power).

Furthermore, the web is to blame for a lot of this. In the 90’s .com goldrush graphic designers became web developers when following the money, but graphic design is only one small part of interface design and very little to do with user experience. However, the n-tiered approach to building apps meant that there was a “user interface layer” which became a detail to be worried about afterwards, (and often by people who would have been failed graphic designers in another life).

I intend to do my best to reverse this imbalance.

When is compiling not compiling

2008-06-24T22:46:00.001-07:00

I was annoyed by a post I read somewhere that talked about shells and compiling and why they don't like compiling etc...

Its 2008, and practically no modern programming tools really can't be made interpreted, or compiled as needed. The differences are fading (eg Java, C++ can all be made to work in shells if you like).

In my day job our rule engine is compiled, kinda. But to make the point that things can be totally interpreted, I wrapped a command line shell around the clips language module:

So there. Its actually becoming pretty handy. Also, Lisp dialects are soooo nice on the command line. Balanced expressions make the perfect unit to ship to the interpreter without trying to guess where it is up to.

STOP THE LOGGING MADNESS PLEASE !!

2008-05-21T16:52:00.001-07:00

A personal pet peeve of mine is the proliferation of logging frameworks. Log4J was fine, but Sun had to go and not use it for the JDK. There is/was commons logging (which was not maintained) and now there is sl4j.

90% of libraries that have logging requirements should not have logging in them. I don't mean that they shouldn't *be able* to log, but they should do it via a callback interface (what would be a "closure" where you pass in a bit of code to be called when a log event occurs).

Instead what we have now are several jars needed to support different flavours of logging. Annoying.

I know lots of people have opinions on logging, and they shouldn't, they are wrong. Its a symptom of the problem known as "what colour should be the bike shed be". (Which is to say if you are thinking about it, you are not really thinking about what is important, and often times people do that because its a form of procrastination - instead of solving the problem at hand they obsess over trivial details).

So please, library authors, just stop it. Now. **

** EDIT: This includes me, I am very guilty of terrible crimes against humanity in this area.

Tech that makes people crazy

2008-05-20T18:13:00.000-07:00

Lately I have been implementing a WebDAV filesystem. OH the pain. The frustration in trying to make it work right on different platforms (I am looking at you, OSX finder). Something about filesystems makes you go insane. Just look at Hans Reiser.

At the other extreme of the stack of technology, we have javascript. Something about JS makes people insane. I have lots of anecdotal evidence of crazy behaviour - DomAPI suddenly going proprietary, and of course the ExtJS fiasco where they tricked people into contributing to an LGPL project (with illegitimate and unenforceable additions to the LGPL licence) and then bait and switch to a GPL licence (upsetting as it took attention away from other toolkits which are under more liberal licences - heck if even started as a layer on YUI).

Well, its open source, so Use the Fork, Luke !

The tyranny of patents

2008-05-18T04:17:00.000-07:00

I am a iphone user, and fan. Its my first experience with a "touch" user interface since I played with old HP machines back in the 80s (one of my school friends' dad was a HP executive and often had different models home to try out, which I loved).

There has been some talk as to why it has taken 20 or 30 years for touch UI to be an "overnight success". I think a big part of the reason is the patents involved: http://en.wikipedia.org/wiki/Touchscreen#Development

Patents are a terrible, terrible thing for innovation. Having an idea is such a small part of innovation, especially in software as well (where the idea is pretty close to the implementation).

JavaOne 2008

2008-05-10T17:02:00.001-07:00

I made the trek across "the ditch" to San Francisco for JavaOne 2008. Its pretty convenient from Sydney, direct flight, comfortable. What is not comfortable is the jet lag. I have never had it so bad, unbelievably bad.

Well the conference was not terribly exciting, but talking to everyone is always worth the effort. Its still very much Sun's conference, not the communities. Sun tend to have their influence everywhere - from their shoe-horning glassfish into all sorts of irrelevant places, and keynotes that "announce" tech that really have nothing to do with the community (JavaFX). That much is disappointing. Even the irrelevant fact that glassfish has a "kernel" of 98K was mentioned in the keynote ! What does that mean ! Nothing ! Its not really a kernel, but whatever. Sun seem to be driven partly out of spite, creating competing tech (or aquiring) - with no aim to make money, but only to compete etc.. Its not terribly surprising they are no longer making money - they had a terrible result recently. Terrible - makes me sad as they were a jewel of silicon valley once, and a good contributor to open source.

I liked JavaFX, would consider using it if it took off.

The most interesting thing all week was the Scala "lift off" conference I went to on saturday. I went with the express aim (and some preparation) to convince people how great it would be for scalac (compiler) to be able to have java and scala source mixed, including in IDE tooling. It turns out it is already on the way !

With such a smart group of people working on and around scala, and funded, it has a bright future. And not a moment too soon.

Melbourne demos

2008-04-06T16:03:00.000-07:00

I was lucky enough to visit Melbourne, Australia's most livable city, and talk to a whole lot of people about drools stuff.

Unfortunately I had a raging cold the whole time, and was barely lucid, but I got to see the new Red Hat office, put faces to names etc. Room service is really nice when you are sick as a dog in bed.

One of the most interesting reflections I had on talking to people, was that open source isn't considered just because its open source (as in the fact that it is cheaper up front) but more on feature for feature merits. This is quite a change from years past.

The demonstrations of the BRMS and other new features went really well (surprisingly for me) - I was really happy with it (as were others).

While there I also joined, remotely, a Dallas user group for rule based tech users. That was quite interesting (James Owen was speaking) - I will definitely join next month as the timezone is perfect for me. Lots to learn...

I still have a nasty cold, but its on the mend, hopefully soon, as at the end of the week I am returning to Sydney (Blue Mountains to be more accurate). Whence I will spend my spare time scanning realestate.com.au and domain.com.au for a house to live in ;)

There presentation I did is here.

Travel and talks

2008-03-24T15:19:00.000-07:00

I am speaking at the local ACS special interests group this Wednesday (the 25th). Next week I will be in Melbourne seeing various people, not sure if there are general meetups though (I have been ignoring email, a bad^H^H^H good habit of mine).

The week after we are moving to Sydney (after 4 years in Brisbane) so the times, they are a-changing.

Now I need to find some stuff to talk about (I really don't like demos - mostly cause I stop in the middle, crack open the code and change/fix something that is bugging me, not so great for the perplexed people watching).

How a sewing machine works?

2008-03-21T04:00:00.000-07:00

As a frequent sufferer of insomnia - I like to think about trivial but unsolved tricks on how everyday things work (how a boat sails against the wind, how a sewing machine works for instance).

Well, now I will have to find something else:

Well if a picture is worth 1000 words, an animated gif must be worth 10^6? I love that this is such a clear explanation to something that would be impossible to describe with words.

Stop iphone opening iphoto

2008-03-17T17:01:00.000-07:00

As a part time mac-tard, and iphone user, it irritates me how it opens iPhoto. I saw some overcomplicated trick at 37 signals blog, but then found out that it is a setting in Image Capture.app ! (preferences of course).

Writing a language in itself

2008-03-02T14:45:00.000-08:00

I have always been fascinated by the art of writing programming languages themselves - in some ways, its an easier problem to wrap your head around. Well perhaps not easier in the technical sense, but in terms of requirements/design etc, mostly you can fit it all in your head at once (unlike solving say, a Real World problem !).

I was curious about the newly minted OpenJDK6 source code that has been released (open sourced, finally), so I ran the trusty sloccount command line tool to count lines, here are the results:

java: 2393565 (72.63%) cpp: 462235 (14.03%) ansic: 419499 (12.73%) sh: 17089 (0.52%) asm: 1936 (0.06%) awk: 598 (0.02%) lisp: 449 (0.01%) cs: 72 (0.00%) jsp: 24 (0.00%) csh: 3 (0.00%)

There is lisp in there? I am betting elisp for an emacs major mode or something. But 72% Java. c/c++ would be for posix interfaces, and probably enough core VM code, and perhaps hotspot stuff.

Conclusion: Java is written mostly in Java, which is the way it should be (if you were looking at smalltalk, I bet it would be much much more then 70%, more like 95%). Compare to MRI ruby, which is 90% C, or JRuby, which is 90% Java.

IBM (and the old PwC) bankrupts 170 year old business

2008-02-13T15:03:00.000-08:00

It is with dismay I read this: http://blogs.zdnet.com/projectfailures/?p=583

I am pretty sure I smell the stench of the Pricewaterhousecoopers trouble makers inside IBM services. Shame.

Should you test libraries?

2008-01-28T02:31:00.000-08:00

Thought of the day: if you depend on libraries behavior, you *should* have some tests to confirm it? (ie not just experiment, but keep some unit tests around to check it).

Sounds sensible to me, but others probably think I am nuts.

The upside: When upgrading lib versions, you know if your assumptions still hold.

The downside: People think you are nuts for writing tests for NONE of you own code !

Well, I like it, I do it, its how I experiment and learn what a lib does, so may as well keep them around as tests (and I am not dead yet).

Still here

2008-01-19T02:40:00.000-08:00

Tend to do more interesting blog entries to do with my day job.

Open Source Developer Conference - 2007

2007-12-02T23:21:00.000-08:00

I was fortunate enough to attend www.osdc.com.au in Brisbane. (Being home city for me, really wasn't much trouble).

It is really a conference for app developers and script kiddies, although there were a smattering of linux kernel die hards. About 40% of the people there were paid (in some form) to work on open source as part of their job. That was surprisingly high to me.

Some general observations:

C is still used for everything, in some way. Nothing is likely to replace this language for a long time. However, if it is in fact replaced, it will possibly be due to the changing architectures of CPUs (getting wider rather then just faster).
I now make the assumption that nothing on the web is secure. There are too many vectors for XSS to make its way into your browsing. People want features more then absolute security. One tip: use a totally separate browser for "sensitive" things, like internet banking etc (and another browser for general browsing - keep your "secure" browser ONLY for those sites, NOTHING else).
FLOSS has under 2% participation rate of women. This is much less then the 28% for the general IT industry. Yeah, its not a great thing. But, of that 28% - I would wager its taking into account larger parts of organisations that would almost never exist in open source groups, so that may exlpain the skew. No one really knows why. I think men are genetically pre-disposed to have an unhealthy obsession with minutia, which helps in this industry.
http://www.oncetechnologies.com/radix/ is very interesting. I will keep an eye on it, if it keeps going, and stays open source, may be something we can build on top of.

So Javascript 2 is the Next Big Thing?

2007-11-20T04:16:00.000-08:00

Well it must have been at least 20 minutes since I started using a completely different language, the latest one is ActionScript 3 (media stuff with flash). Which is approximately Javascript 2.

All find and dandy, but you get a type warning on this:

var boo = true;

Really, come on. Surely there is *some* merit in the most obvious literal type inference. There is none, nada. I know there is some debate on the appropriateness of type inference (mostly due to the fact that if you or the compiler get something wrong, the error messages can be impenetrable), but surely its safe enough for literal vars.

Rails on JBoss

2007-10-24T23:38:00.001-07:00

Finally had some more time to do some more work in rails. I heard about "goldspike" - a framework to use jruby to allow you to get rails apps into a war, and deployed on an app server.

I am reproducing the very very very simple instructions on how I did this mostly for my own benefit.

Step 1: hack your rails app, just as normal
Step 2: gem install ActiveRecord-JDBC
Step 3: ruby script/plugin install "svn://rubyforge.org/var/svn/jruby-extras/trunk/rails-integration/plugins/goldspike"
Step 4: rake war:standalone:create

Thats it ! Well there are a lot of optional configurations, but that will generate a war you can drop and run immediately. No need to get jruby separately.

I may have a concrete use for this very very shortly.

Dynamic Language Wizards

2007-10-24T03:33:00.000-07:00

I enjoy looking into the history of programming languages (tragically, its one of the few things that is a hobby of mine). I came across this video (2001) about "dynamic languages" - a panel of gurus. Its the only time I have seen Guy Steele and Paul Graham in the same room. Its quite interesting and has held up very accurately over time. Well worth a watch: click here.

Another reason why we all suck.

2007-10-23T02:06:00.000-07:00

I thought XUL from Mozilla was dead (although I really liked its idea) but someone showed me a working rich UI. Oh man, everything else kinda sucks.

Scala Scala Scala & why it can do what dynamic languages can do

2007-10-09T16:07:00.000-07:00

Recently I have been doing some research and study into the Scala language (and also some side stuff with Haskell to learn and refresh some concepts in my mind which are related).

I did a talk on Monday night at the local JUG (www.qjug.org) in terms of presenting scala gently to people NOT from a functional programming background (specifically, people who know Java or dynamic languages). I have attached the presentation for this (some of the jokes won't make sense without the talk but it wasn't recorded).

The 2 interesting parts of it were how do to "duck typing", and how to add in functionality to existing objects - these are both things that people like to do with dynamic languages. I showed how you could do this in scala without escaping the type system, and how neat and easy it could be.

"Duck typing" - that one is easy, thanks to Scala 2.6 you have structural types (basically declare a method parameter as requiring certain methods and thats all you need to do).

"Decorating" existing objects - that was not quite so obvious. Using "implicit defs" in scala allows you do to this - the compiler will see that you are trying to access a method on an object that doesn't exist, and it will see if there are any implicit defs in scope that can convert the object to another object that does implement the method (all using the type system).

Well, I don't really feel like repeating myself right now so to make more sense of that you will need to look at the attached presentation.

I think there is a lot of work to be done in educating people whos only experience of type system is what mainstream languages (C#, Java etc) provide, and showing them how it was "meant" to be without throwing too many concepts in their face.

Presentation is here.

Python decision tables in under 77 lines

2007-09-24T04:12:00.000-07:00

A colleague at work heard I knew a thing or two about rule engines, asked if I could help him build something to triage his support queues. Working where I do, python is kind of a Big Thing, and I knew he was a big fan of the Django web framework, so I thought I would have a crack at doing some python (a very simple decision table implementation). I tried to learn and follow python idioms, and basically ignored the python OO style, instead going for a quasi functional approach (which python shines at, its OO feels kind of odd).

Key to this are map() and filter(). Python has lots of built in functions. This makes it a little hard sometimes to find where something is (if its a class method, its easy just >dir(classname) will give you a list of methods - but you sometimes forget what is a build in function or what is a method, but anyway).

map takes a procedure (function pointer ) and a list of things to do stuff over (in no particular order), and produces a list, filter is similar, but only returns items where the procedure returns true. I think map isn't truly functional, as it operates on the list in place (there is no deep copying going on like in haskell). The lambda in python is pretty limited, I would say useless. But no matter, you can "nest" function definitions, and then you just pass the name of the function.

Anyway, the code here. I was amazed I was able to do it in under 77 lines (including a few comments). Python is nicely terse, but the enforce whitespace makes it very readable. And the functional style is certainly compact. And I think it looks pretty. Currently parses XLS (but I intended to allow other front end parsers).

Scheme and how the world has gone crazy

2007-09-15T03:19:00.000-07:00

I have been spending time working through the videos of SICP (you can get them on google video) - from MIT. I like to follow along, pause, and try and solve the problems in Dr Scheme or GUILE before they show the solution. The course uses Scheme of course (which they call LISP - although since that time, most people think you mean common lisp when you say lisp, but scheme has just as much or perhaps more right to be called "lisp" then CL does).

What is interesting is watching it in 2007 and realise that since that time (this was recorded in the early 80's, but the course goes back even further), is how really nothing much has progressed in software at all. Sure things are faster, and we have new applications (like the web) which are amazing compared to at the time, but in terms of art/engineering/science of creating software, we have gone backwards.

Some interesting points:

The concept of "DSLs" was pretty clear in the early lectures. They go far as to say Lisp is a bad language to solve your problems in, but its a brilliant language to build a embedded language which will allow you to solve you problems now and into the future (ie language abstractions, rather then fussing with APIs and up front design).
The foolishness of the divide and conquer mentality and up front design (ie basically most mainstream "methodologies" and the foolishness of a using a traditional process in coming up with solutions). Yet still people try on the same old ideas, over and over, and still get the same results, and still are surprised.

There was so much achieved from the 60's to the early 80's, yet since then, not much. We went from not having the concept of a programming language (or barely) to using AI techniques to get computers to design VLSI chips. Now what do we have? All it takes to be "innovative" these days is a new framework for CRUD. Or a website that allows you to do what IRC did in the 80's.

Overall I feel depressed about the whole situation. The day may come where its too much and I will just give up.

Espresso & stuff

2007-09-05T01:42:00.000-07:00

For working at home, a mans best friend is his espresso machine. Gaggia Baby D. Accept no substitute. Only italians and germans should be legally allowed to make espresso machines.

I am slowly but slackly migrating my personal web content to www.michaelneale.net.

Enabling scroll wheel in fedora

2007-08-13T22:21:00.000-07:00

This is a slightly mindless post - it is really a tip that I am recording so I can remember, and perhaps will help google find the answers.

I am currently heavily using VMWare Fusion on OSX, and running Fedora 7 (no core any more !). This works great, but I had hassles getting the scroll wheel to work on the mouse (this includes double finger dragging on the macbook pro - see previous post for photos !).

The magic trick is to crack open xorg.conf (as root - in /etc/X11) and find the section on the mouse, and add to the section "InputDevice" that refers to the mouse:

Option "ZAxisMapping" "4 5"

Ahhhh.... well thats why well all love X :(

but anyway, it works.

New Toys

2007-07-22T03:20:00.000-07:00

I finally for the first time every bought *myself* a mac - a macbook pro. So, I thought I would celebrate the new arrival by letting my 3 and 3/4 year old open it, and start it up. How hard can it be?

The box is really really easy to open:

Mm... the narcotic smell of the apple factory... see the eyes?

Lots of bits. Chloe's favourite component is the remote (of course).

Google Developer Day 2007 (Sydney)

2007-06-28T00:52:00.000-07:00

I managed to get to the Google Developer Day mini-conference in Sydney, was well worth the trip, and much better then a poke in the eye.

Its interesting that in the past it was Sun, Microsoft, Oracle etc hosting these sort of corporate sponsored developer events, now its google that seems to have taken the mantle. It was a great turn out, a surprising amount of energy. Its now clear to me that Google are one of the New Platforms (perhaps the biggest one) that makes operating systems, if not irrelevant, then less central then they were. The network truly is the computer (finally).

My take aways:

Atom is an important protocot. Its behind all GData stuff. Atom + REST == good ! I wish I had time to look more into Atom, but it looks like its based on concepts that I agree with.
If you want to write apps for The Google Platform, Gadgets are the way !
Google gears was announced - 'nuf said ! The interesting thing for me was their worker pool and message based concurrency for multi threading - that will result in much less broken apps then modifying Javascript itself to cope with threads. I like that.
Lots was said of the greatness of GWT - I already knew all that (as a user) - GWT is something I thought I would hate, but it seems to work well in all the right ways (it has shocked me). Google are very keen on this (especially for their larger team apps). It was interesting that Java was chosen entirely for tooling, not cause its a great language (as a language it has lots of well documented failings - which are slowly being addressed - not fast enough for my taste though). It is a users first toolkit (not developers) - its hard to make an unresponsive GUI with GWT - and CSS for layout and design is a sweet spot (although I am not that great at it myself).
Google want to make targeted, isolated strategic improvements to the browser, not replace it. Hence the gears design approach. I like that, well, actually I don't know any better but I would guess Google know what they are talking about.

Here is a happy snap of me (on the left) and Shane Bryzak (from JBoss) on the right:

creating languages and syntax error messages

2007-06-24T20:05:00.000-07:00

One of the hardest things I know from experience about creating new languages, is how to report error messages (syntax) in a helpful way (ie reduces peoples urge to hunt down and kill you).

The current trend to create DSLs willy nilly (yes, I like using the term willy nilly) is that there is scant regard for this - I hope this doesn't mean there is a legacy of apps out there which will doom their maintainers to years of pain and frustration (I am afraid it will). DSLs are really nice, but they do cause frustration.

Unfortunately, I don't really know the answer to this. I know I have my favorite compilers in terms of nice messages, and my least favorite (good == most python interpreters, bad == ruby, good == JDT from Eclipse, bad == any open source Common Lisp implementation I have tried to date, good==guile - its scheme, but it doesn't freak me out when I do something silly).

I wish I know what made things good or bad. Maybe its taste - I tend to prefer syntax error messages which are less jargon and more plain language explanation, often with suggestions. I guess if you have and take the time to master a language, it kind of blends into the background, so perhaps thats why people don't tend to focus on it as much as I think they should.

Ruby talk - RubyGnome2/GTK

2007-05-28T15:32:00.000-07:00

I spoke at Brisbane.rb meetup recently (Brisbane Ruby Brigade). I did half the talk - showing off what you can do with RubyGnome2.

As promised, here are some resources/examples for getting started with it. Lots of the examples out there neglect the "Gtk.init" call, which is annoying, probably stops a lot of people in their tracks, but in any case, the following should work well.

I found that a lot of the examples on the web either didn't work as advertised, or just had typos, or were missing Gtk.init call. So here I my examples that I promised worked for me ! (and I have witnesses to prove it !). You will need the ruby GTK2 stuff installed, and libglade2 for ruby as well.
The presentation + example files are here.

I keep some handy articles on del.icio.us/michaelneale/ruby+gtk.

For a really simple example, my "DSL" for a form was:

input_data = {
:name => "Michael",
:age => 42,
:topping => "Chilli",
:drink => "Beer"
}

popup_dialog :data => input_data,
:icon => "hoff.jpeg",
:actions => {
:quit => lambda { Gtk.main_quit },
:dump => lambda { puts input_data.to_s }
}

Jython is alive and Kicking !!

2007-05-23T23:34:00.000-07:00

Mark (Proctor) got an email from Frank Wierdbicki - who (as it turns out) is a colleague at Red Hat. He is the Jython lead. For some time now I have heard (and probably propagated the myth) that Jython has kind of petered out.

That is not so ! Frank has a blog, and is actively getting Jython up to speed for a new release. They had to play some catchup with CPython (the official version anyway) and some major re-engineering meant it looked like they went quiet.

But they are on the case, and Frank seems like he is on this full time, which is great. He also mentioned that he has been in touch with Charles Nutter from the JRuby project to see what dynamic JVM tricks that can share, and how to make the JRE a better environment for dynamic languages.

I have a growing affection with Python of late (nothing to do with Java) so this is also good news for me.

AJAX frameworks are NOT pointless

2007-05-07T22:44:00.000-07:00

The following was from a Slashdot post, incredibly detailed, but is a great analysis on why ajax is harder then meets the eye, and why you should use a mature framework for it.
(from here.)
--------------------------------------------------------------------------------------------

AJAX frameworks are NOT pointless

by francium de neobie (590783) on Thursday May 03, @10:24AM (#18972115)
(http://martinkou.blogspot.com/)

There are many little funny things that just happens when you're coding a web application in JavaScript without a framework/library/toolkit helping you. Unless you're really an AJAX/JavaScript wizard, coding an AJAX-enabled web application on your own and mixing online code receipts is a very dangerous thing to do.

Browser inconsistencies
This is the most obvious one, but only the entry to the rabbit hole. If you are not familiar with the example (maybe not exactly the same, but any AJAX web developer worth his salt should have seen one like that) I give below, then please, PLEASE, do yourself, your fellow developers and your users a favor, resist the urge to hack things together for once, use a mature AJAX framework.

An important part of AJAX is that you need to update what is displayed on the web browser in the client side (by JavaScript), without refreshing the page. This implies that you're very likely to have to create and destroy DOM nodes on the fly. Now, how do you create a radio button in JavaScript?

How about...

var node = document.createElement("input"); node.type = "radio" node.name = ... node.value = ...

That's what you would do if you follow the DOM standard. But sorry, this does not work. Try to create a radio button with the above code segment in Internet Explorer 6, you'll get a broken radio button - you can't select it. The correct way to create a radio button by DOM manipulation is described in this MSDN article [microsoft.com]:

newRadioButton = document.createElement("")

Memory leaks
The last one was easy. Do you know you can make a web application that leaks memory like a sieve in Internet Explorer 6 by making a simple circular reference like the following one?

var node = document.createElement("div"); node.someAttr = node;

If you're a good programmer, I might have sounded an alarm in your head right now - any circular references involving DOM nodes in IE6 results in memory leaks that persist after URL changes or page refreshes - unless you use an AJAX toolkit that takes care of the issue for you. Have you assigned a DOM as an attribute value under another DOM node in the past? Yes? Then you'd better check your web application for memory leaks with Drip [outofhanwell.com], now.

What's more, it's not just assigning DOM nodes as attributes that would result in memory leaks, closures in JavaScript can also form circular references and cause memory leaks. What makes closures particularly dangerous is that circular references with closures are not easy to spot. For example, the following code segment leaks:

var node = document.createElement("div"); var clickHandler = function(){}; node.onclick = clickHandler;

Looks innocent enough, but you've already formed a leaky circular reference here. node->clickHandler->node.

For more information about memory leaks under IE6, read these:
Mihai Bazon's blog entry [bazon.net]
MSDN's lengthy and confusing description of the problem [microsoft.com]

The XMLHttpRequest object is not as simple as you think
Much of the magic of AJAX comes from the XMLHttpRequest object (or its ActiveX equivalent, or an iframe, etc.), right? Sure. If you're only doing something simple via AJAX (like, updating the server time), then you can just copy an XMLHttpRequest code snippet from sites like this [apple.com] and hack away, right?

Wrong! Those XMLHttpRequest code snippets are one of the very reasons why people are thinking AJAX as a hack - it sometimes doesn't work! The XMLHttpRequest code snippet given on Apple's site can be broken in commonly encountered situations, and you can simulate that yourself:

Write a simple AJAX web application that retrieves and displays the current server time on a web browser using Apple's code snippet.
Test it yourself under normal conditions. So it works and it's safe to use, right? Let's see...
Change your computer's routing table such that you can have no route to the web server.
Now test your application again in Firefox. Your application should fail. But does it fail gracefully? No. You see an error message in Firefox's error console stating that the XMLHttpRequest object's status attribute cannot be read. If you have coded something to handle AJAX request failures, your handler won't be called.

Why is that happening? It is because, any socket errors happening during an AJAX request will cause the onreadystatechange handler to be called under Firefox, yet the status attribute cannot be read. Reading it causes a JavaScript error and stops JavaScript execution (unless you add a try...catch... block there, but that assumes you already know about the problem so it's moot)! Under Internet Explorer, reading the status attribute in the same situation gives you the socket error code instead. Don't know about these stuff? Please, use a mature AJAX framework.

Performance problems
Coding AJAX applications is just like writing things in C++ or Java - so long as you're using efficient algorithms, your application should run fast, right?

Of course, you are wrong again. Let's say... in some part of your application, you want to concatenate a lot of string fragments together to form a long string in a for loop, how do you do it? How about...

var targetString = ""; for(var i=0;i

That's the way most programmers would think of, intuitively. But the performance of that sucks under Internet Explorer. The correct way to combine strings under JavaScript is to use the Array.join() operation. You can read more about this here [comet.co.il]. The optimization I talked about is also implemented in Dojo Toolkit (kudos to Alex Russell), and I believe any reasonably robust AJAX framework should have it too. Not knowing about such problems, had you hacked together a fairly sophisticated AJAX web application yourself, you would be running into performance hell sooner or later. Taking 646ms to combine strings still doesn't sound very slow for you, right? There are many more performance traps in JavaScript. Do you know there's a very significant performance difference between the following two code snippets? First code snippet:

// placing 5000 "Hello World" messages in random positions for(var i=0;i<5000;i++) node =" document.createElement(" position = "absolute" left =" parseInt(Math.random()" top =" parseInt(Math.random()">

Second code snippet:

// placing 5000 "Hello World" messages in random positions for(var i=0;i<5000;i++) node =" document.createElement(" position = "absolute" left =" parseInt(Math.random()" top =" parseInt(Math.random()">

The only difference between the two code snippets is the placement of the document.body.appendChild() line. But if you actually test them out, the second code snippet is much faster, under both IE and Firefox. The performance difference has nothing to do with your algorithms - you just shuffled one line of code around; it has to do with how the browser render the randomly placed DIV nodes. Ever wondered why your hacked together web application is taking half a minute running JavaScript after all the files are loaded? So, unless you're already a programming god or don't mind spending lots of time solving bugs that you shouldn't have solved; you really, really should use some of these AJAX frameworks if you're making anything fairly sophisticated with AJAX. Disclaimer: I'm also a developer of an AJAX toolkit myself, but I did that for my final year project (project group DE3) [cse.ust.hk] in my university. So it's more a project for fun rather than a serious contender to industry strength toolkits like GWT or Dojo. My project website: WT Toolkit [sourceforge.net].

Rules with Spring

2007-04-13T05:49:00.000-07:00

Its odd, normally google searches work quite well into our Wiki (JBoss Rules), but for some reason when serching spring, it doesn't show up this.

Is.This.A.Type.Or.A.Method.Or.What ?

2007-04-06T21:36:00.000-07:00

The title of this post is a comment on the stupidity of Microsoft's contrarian coding standards they adopt for C# in particular, just to avoid camelCase, or even acknowledging that Java ever existed.

The Language of the Year

2007-04-04T15:40:00.000-07:00

For some time now I have tried to follow the dogma of the Pragmatic Programmers.
Specificially, the learn one new programming language a year (I am sure I am not the only one to notice the irony of the increasing dogma that the PP guys are emitting, but it doesn't really bother me too much, they need to sell books, and their advice is great anyway).

So 2005 was the year of C#, well that took all of 10 minutes.
2006 was the year of Ruby - I feel I have come close to mastering Ruby - mainly I tackled the meta programming and other areas that I found trickiest, and the rest seems like gravy.
2007 Lisp

I have been a ruby fan since 2000, since it became "cool" around 2004 I kind of moved away from the hype though, but things have calmed down a bit. Ruby is an easy language to use to get stuff done. Recently I have been also dabbling with python for the same things to keep it fresh (please don't hate me !).

But, 2007 is the year of Lisp (2006 I dabbled, as I had to, to learn a sub dialect of Lisp - CLIPS). My attention is wavering towards Haskell though, I feel the need for functional purity to clean my mind, as Lisp seems to be capable of anything, it could well blow my mind. But, so far, so good.

Not many people know that the first programming language I learnt properly was Miranda, and I hated every second of it. Many people theorise that we could get better teaching results by teaching student FP first - well, I am only a single datapoint, but it didn't work that great for me. It scared me off the "purer" aspects of CS and more into engineering, where C was a higher level tool for when you couldn't handle the ASM (and C made soooo much more sense, as did pascal, modula 2, anything imperative). So for me, the FP myth was busted, but as I said, just a single data point.

But now I am ready, and besides, a need an academic excuse to get that multi core laptop I want.

Trademark hyperbole

2007-03-24T21:22:00.000-07:00

The dark clouds of a FUD storm have appeared, and rained, regarding Red Hat sending out a letter about Hibernate trade mark infringments.

Below I have a quote from Mark Webbink, our friendly general council pimp (thats a lawyer - a law-talking guy).

Basically to spell it out clearly, here it is:

NAUGHTY USAGE OF TRADEMARK:

Hibernate Training

GOOD USAGE OF TRADEMARK:

Hibernate (R) Object Relational software training

Yes, the differences are subtle, get over it. Even the Apache Foundation will defend its trademarks in the same fashion. An guess what, law is subtle, and fussy about syntax - just like a compiler !

You can of course become a partner and then you are licensed to use the trademark fully- its part of the value of being a partner.

OK, well Mark's response appears below in full:

"I am writing to clarify the issues raised by the publication of Ms. Robertson's communication on behalf of Red Hat. First, the letter is not placed into the context of the situation it was addressing. That presents the opportunity for misinterpretation. At the same time, I would agree that the letter is less than precise in defining what has been done wrong and the corrective action that is required. Ultimately, that is my fault as the person in charge of trademark enforcement at Red Hat.

Contrary to Gavin's statements above, you cannot offer HIBERNATE Training or JBOSS Training. This is an improper use of Red Hat trademarks in that the marks are being used (a) either as nouns or (b) to promote a good or service that is directly branded with Red Hat owned marks. What is permissable, and I am sure this is what Gavin meant, is that you are permitted to offer HIBERNATE(R) Object Relational Mapping Software Training or, as another example, JBoss(R) Application Server Training. Here the marks are being applied to the goods in a proper manner and it is clear that the training is being provided for that branded technology, not by the brand owner. As a further common courtesy, it would also be appropriate for those properly using the marks in this manner to make clear that they are not in anyway associated with Red Hat or its JBoss Division.

With that clarification I hope I have resolved the confusion and/or discontent around this issue. More extensive information on the permitted uses of Red Hat marks can be found at http://www.redhat.com/about/companyprofile/trademark/

I would also ask, as a courtesy to Ms. Robertson, that the party who posted her letter please indicate that they were the party posting the letter, not Ms. Robertson.

My apologies for any confusion that has been caused. "

Mark Webbink
Deputy General Counsel
Red Hat, Inc.

OK - everyone happy now? Defcon 1 again? calm down, take a deep breath. Nothing to see here.

The future of storage

2007-01-20T20:30:00.000-08:00

Is here.

And I survived the trip back home.

Relocation relocation

2007-01-14T12:47:00.000-08:00

Well its time, have had a blast in London. But time to return to the land down under. See you all on the other side (man I hate flying).

Voice quality on phones

2007-01-05T01:24:00.000-08:00

I am a little old fashioned, and still occasionally like to use a phone for, well, speaking to other people. My wife is nagging for a black berry (I am avoiding them), and I think the mobile tech is great.

But, why is voice quality so bad? Someone decided long ago, that only 300Hz to 3kHz was needed to reproduce the human voice for verbal communication. BOLLOCKS ! This was an engineering compromise with the amazingly primitive tech they had at the time. Yet nothing has changed. It kind of makes commercial sense these days to limit bandwidth, but still, why can't I PAY for MORE quality? I want to talk to someone and hear every nuance and sigh as they smirk at my clueless-ness.

I would be happy to pay a premium to be able to have premium quality voice connections when I need it. Otherwise, I just prefer IM and email, communication over the phone is futile.

Climbing the charts

2007-01-04T01:37:00.000-08:00

Was doing some ego surfing, and I noticed my long time home page (from my trusty ISP internode) is climbing the charts when I do a google search for Michael Neale, which is good, in case I go all Momento or anything.

migrating here

2006-12-13T13:33:00.000-08:00

I think I will move all my personal blogging stuff to here.

At present all my relevant (non) personal info is here.
Blogspot is quite nice to use, so am happy to move to it (but I still like a olde fashioned vim friendly home page, you can pry that out of my cold dead hands).

What with Christmas coming up (in Paris this year) and then moving back to Sydney, will be quiet for a while.