/dev/neal

ECMAScript test262 - Awesome, But...

2011-07-01T08:42:00.003-04:00

Today the IE blog linked to ECMAScript test262, a test suite to verify conformance to the JavaScript spec.

First, kudos to ECMA for coming out with a test suite - having a benchmark for compatibility will drive the browser vendors to improve their interoperability. Everyone wins there. Also, the visual design of the page is very nice - something we don't always get from standards bodies.

Second, kudos to the IE team for getting the top score with their IE 10 preview! And IE 9 is way up there also. Seriously, good job guys. We know that I have my problems with Microsoft's browser policies, but nobody can deny that the IE team is doing great work over there.

Now, the bad news (emphasis mine):

Test262 is being developed by the members of Ecma TC39. Ecma's intellectual property policies, permit only Ecma members to directly contribute code to the project. However, a public mailing list is used to coordinate development of Test262. If you wish to participate in the discussion please subscribe. Bug reports and suggestions should be sent to the mailing list.

(I can't link you directly to that quote because they didn't use a separate URL - it's revealed by JavaScript when you click on the "Development" link in the top navigation. They didn't even use a hash-bang link!)

If those are your policies, guys, change your policies. This is exactly the kind of thing that should be open source. You're not trying to sell it. Lots of talented people have incentive to contribute to it.

Why do developers hate IE?

2011-06-14T23:36:00.002-04:00

It has to be a tough gig being on the Internet Explorer team at Microsoft. Mention IE in a room full of web developers and the best reaction you can probably expect is a sigh. You’re more likely to see fists shaken at the sky or hear muttered curses. But why?

Some people explain the IE hate as just part of an anti-Microsoft prejudice. That’s sort of a non-answer. Why do people have negative feelings toward Microsoft? There are reasons for those anti-Microsoft feelings. One of those reasons is certainly just discomfort people have toward large companies that have power over them. But those kind of feelings apply to Google and increasingly Apple as well. In the case of Microsoft, though, web developers can look back at an incident where that power was wielded against something they loved - Netscape.

The Ghost of Netscape

For many web developers today, Netscape was their first web browser. Think about that. Netscape was the web at a time when a lot of us fell in love with the web. What are we today? Web developers. Turns out we liked the web quite a bit. And we liked Netscape.

Microsoft killed Netscape in a way that seemed very unfair.

Microsoft bundled IE with its near-ubiqutous operating system. This was in the bad old days of the “Browser Wars” when there were no web standards to speak of. If you wanted to do anything fancy at all (and standards for “fancy” were much lower), you had to think very hard about which browser you were on. Microsoft’s move to tie the browser to the OS said: “Target what you want, but everybody’s going to have IE. They might not have Netscape.”

It still pains me to say it, but Microsoft pushed the state of the art for web browsers forward a lot in those years. A lot of the great browser features we have today came from IE. Internet Explorer became the better browser. But what we remember is what Microsoft did to Netscape. We remember how they betrayed our trust and killed something we loved.

But that’s all in the past, right? Ancient history in Internet years. Microsoft got slapped by the DoJ and some European anti-trust stuff also - they’ve done their time. What about the IE of today? Well, not so fast.

The IE of Yesterday

Microsoft had won the browser wars by the time it released IE 5.5. Netscape 5 was never released. When Netscape gave its source code to the community, the community decided: “uh, let’s just start from scratch.” Netscape 6 was released as just a re-branded copy of a pre-1.0 version of Mozilla. It was buggy and a resource hog.

It’s not surprising that Microsoft released IE 6 and declared victory. Netscape had been vanquished. Microsoft had proven that they could come from behind and create a better browser. I imagine there were ticker-tape parades in Redmond.

The year was 2001. Little did anyone suspect that the year would stay 2001 for more than a decade.

Support Cycles

Complex software tends to have bugs. The more complex, the more chance for bugs. Bugs in network-connected software tend to get exploited and turned into security holes. The web browser is simultaniously one of the most complex and one of the most network-connected pieces of software on any given computer. If you don’t keep your browser up-to-date with the latest security patches, you’re going to get exploited.

So Microsoft supported IE 6. Patches came out at regular intervals. But they were mostly just security fixes. Microsoft wouldn’t fix rendering bugs because they didn’t want to rock the boat. Sites out there depended on the sometimes quirky (to put it politely) rendering that IE gave them.

Meanwhile Mozilla hit 1.0. Then Firefox grew out of Mozilla. Then it turned out people really liked Firefox. Even Apple got in to the browser race with Safari. Microsoft decided they had to step up their game, so they got the team back together and came out with IE 7 around the same time Firefox 2.0 was coming out - late 2006.

IE 7 was a huge improvement over IE 6. It had a tab-based interface that had become popular - that was nice for users. Developers cheered because a lot of the rendering bugs they had been tearing their hair out over with IE 6 had been fixed. “Finally,” we thought, “these IE 6 bugs we have been wrestling with for five years are no more!”

Microsoft put IE 7 out through Windows Update… but as an optional update. Optional? It turns out that some “enterprises” didn’t want to update to IE 7. They all had internal apps that depended on the rendering bugs in IE 6.

And enterprises weren’t the only IE users that stayed on IE 6. Lots of Windows users never consider installing optional updates. They’re barely comfortable installing the required updates - an optional update just sounds like an invitation to break their computer.

So at the end of 2006 Microsoft was supporting two browsers - IE 6 and IE 7. And that meant that web developers had to support IE 6 and 7 also. All our cheers at the release of IE 7 turned to cries of frustration.

The pattern repeated when IE 8 came out in 2009 - optional updates. Microsoft started supporting three browsers - web developers supported three versions of IE.

Earlier this year Microsoft released IE 9. It is a huge improvement over IE 8. It supports nice things like box-shadows and rounded corners. It has a vastly improved JavaScript engine. It is still an optional update for all IE users. Actually not all IE users - users on Windows XP can’t update to IE 9 - the browser requires Windows Vista or 7. Now we are all supporting four very different versions of IE and nobody is happy.

Why is IE 6 still supported?! It is apparently Microsoft’s policy to keep supporting the major browser version that was current when the operating system was released. For Windows XP, that browser was IE 6. And when will Windows XP support end? I think it’s not until 2014. At that point IE 6 will have been on the market for thirteen years. Thirteen years!

This should have never happened. Microsoft bowed to the wishes of some of its big business clients by supporting old major releases of IE. In doing so they decided to screw over web developers. Does that piss us off? You bet it does!

When IE 7 came out, IE 6 should have ended. You want a security patch for IE 6? Here - it’s called IE 7. Sure it would have been painful for a lot of companies, but guess what - supporting multiple versions of IE is a hell of a lot more painful to a lot more companies. And those companies that still haven’t broken out of IE 6-land? They are in for one hell of a rude awakening when XP support ends. All this “don’t rock the boat” policy has gotten us is a really unstable boat.

And Microsoft hasn’t changed their policy. They’re still going to support IE 7 until Vista falls out of support. And IE 8 until Windows 7 is no more. How long will that be? How many different versions of IE will we have to worry about?

Make no mistake - IE is a giant dick in the eye of all web developers. And Microsoft has chosen to make it so.

How can they fix it? They can stop supporting major releases for so long. Just support the latest version! What the hell is a “major” release anyway? Google increases the version number of Chrome any time someone sneezes. It doesn’t matter. Call the next version of IE 9.0.1 and make it a required update to all 9.0 users! Do not make an IE 10 that you support along side IE 6, 7, 8 and 9! What the fuck is wrong with you people?

A Big Problem

Remember Steve Ballmer’s “Developers, developers, developers” speech? He was right. Developers are important. It is not good business for Microsoft to be hated by the developer community. And guess what - a large percentage of developers are web developers. And Microsoft has fucked us over for years.

Could they make us happy now? I don’t know. Maybe Microsoft’s best strategy would be to continue fucking us over forever. Eventually all developers will hate Microsoft and never build on one of their platforms again. But in the meantime they can squeeze all their locked-in customers to the fullest.

I really can’t imagine a Microsoft that could win me over now. A more useful web just makes Microsoft’s main products less important. Why would they help it? But the web is moving forward weather Microsoft comes along or not. And in spite of anything Microsoft does, it’s a wonderful ride.

Vim-Columbus

2011-06-03T07:48:00.002-04:00

I'm really excited about the new Vim user group we're starting up in Columbus: Vim-Columbus. I started the page while we were talking about it at Code and Coffee yesterday. That evening after work, inspiration struck and I decided to make the page itself look like Vim. I am inordinately pleased with myself (I get like that sometimes).

Is Ruby the Best Language?

2011-03-12T07:23:00.005-05:00

I was noodling around with some Ruby the other day, and all of a sudden I ran into a situation where I wanted to do a partial function application. But how do you do that in Ruby? It can certainly be done, but it felt very awkward to me and made my code less readable than the non-partially-applied alternatives. So I didn't do it.

For me, higher-order functions in Ruby are the language's only blemish. It seems almost unfair to ding Ruby for that, because Ruby (at least since 1.9.x) does higher-order functions pretty admirably. It's just not as elegant as the rest of the language.

Ruby has set an extremely high bar. The lack of ceremony in the language makes it extremely readable. The purity of its "everything is an object" stance makes it very writable. And its pervasive methods of introspection make it amazingly powerful.

But why did I even have the urge to make a partially applied function? Because I've been doing a lot of JavaScript lately, and JavaScript is all about the functions.

Ruby and JavaScript take sort of opposite approaches to object properties vs. methods. When you want to reach into an object in Ruby, you always talk to a method. So-called "properties" on Ruby objects are really just getter and setter methods with a little syntactic sugar applied.

In JavaScript, objects mostly only have properties. A "method" in JavaScript is just a property that happens to be a function. (You can define real getters and setters in modern versions of JavaScript, but browser support is not yet pervasive.)

Which approach is better? I would have to say Ruby's. JavaScript seems more fragile in its property-centricity. For example, if you want to pass a method as a callback in JavaScript you have to think about weather it needs to be called as a method or if it is OK to be called as a stand-alone function. If it needs to be called on a particular object you end up wrapping it in another function.

When it comes to JavaScript's advantage - higher-order functions - Ruby sort of cheated. By making a special case of passing a single function (a block), Ruby made the 80% use case extremely easy and simple.

So is Ruby the best language? Well, I probably wouldn't have worried about higher-order functions so much if I wasn't using JavaScript. What other blemishes might Ruby have that I just never run into? I guess it's not surprising that you can only compare languages that you have really used.

But Ruby is definitely the best language that I know.

node.js

2009-12-03T23:41:00.003-05:00

I can't stop thinking about node.js.

node.js is Google's V8 JavaScript engine tied together with some non-blocking IO libraries. Those libraries include DNS, TCP, HTTP and file access.

Now, I've been dreaming about a good server-side JavaScript solution for quite a while. JavaScript is an excellent language, and chances are you're using it already on the client side. Why not use it on the server side also? That's not the main reason that node's author (Ryan Dahl) chose it though. He chose it because JavaScript is already very callback-centric, and node is designed to use callbacks in place of multi-threading. Whenever you do something that might take a little while (like hitting the network or disk), you set it up and then pass it a callback. Your callback will know how to handle the data that gets returned. Meanwhile, code keeps executing. This is much better than having one thread running for each connection, because those threads are usually waiting for the network or the database or even disk. When you have all those idle threads you pay a penalty in context-switching and memory overhead.

If you still need more convincing, check out Ryan's slides from his JSConf.eu talk (PDF). They're very readable slides.

It's still early days for node.js. The project (what do you even call it - a server, I guess?) can serve up HTTP very well, but it doesn't give you a framework by itself. Consequently there are frameworks popping up all over for it, following all the patterns of frameworks implemented for other languages. One thing the server is genuinely missing is database support. A PostgreSQL binding is in the works, which sounds great to me. MySQL might be harder because it doesn't have a non-blocking library, but who needs MySQL when you have Postgres? :-P

Network Printer

2009-05-29T13:39:00.002-04:00

If you're trying to set up a printer that is attached to the network, do you think you should select the option that says "Local printer attached to this computer" or the option that says "A network printer, or a printer attached to another computer"? Hint: this is Windows.

Scroll down to the bottom of the dialog box for the answer. I can't understand why they don't just relabel the options. I would make a guess, but Al Gore tells me that my sarcasm footprint is already too high.

The Forgotten Half of Computer Literacy

2008-10-15T11:48:00.004-04:00

Gruber today noted that the New York Times is adding an API to their campaign finance data (bravo!). He also pointed to an article from 2006 titled "The programmer as journalist: a Q&A with Adrian Holovaty".

The interview is pretty concise and lays out how and why journalism can better be accomplished with help from software. Holovaty breaks down the journalistic process of collecting, filtering and disseminating information and how computer automation can be applied to each step, leading to better journalism. Near the end, he argues that all journalists should have at least some experience with programming, if for no other reason than just to know what is possible.

His points are persuasive, but they're also applicable to almost any field - not just journalism. I believe that everyone should at least know how to write software in the same way that everyone should be able to write in general.

Literacy describes the ability to read and write. When we talk about computer literacy, we usually think of the ability to use software - which I would liken to the "reading" part. We often ignore the "writing" part - the ability to make software. But writing software is incredibly powerful. If you write a page-turning novel, it inspires the imagination of everyone who reads it. If you write an informative article, all its readers are enlightened. Likewise, great software entertains and/or empowers every person who makes use of it.

The analogy holds up pretty well. No - it holds up too well - it's not an analogy at all! Writing software is writing. It might have an apparently arcane grammar, but making software is merely writing down instructions that a computer can understand. And the language is not more complicated than English. Have you seen English? It's the most bastardized, crazy language on the planet! The languages we use to talk to computers are incredibly simple - necessarily so. Computers aren't going to pick the correct meaning of a word out of a dozen possibilities based on context. They're not going to detect our sarcasm in step two of the instructions we give them. They need to be told exactly what to do in the simplest way possible, and programming languages are the way we do that. Consequently, programming languages have only gotten simpler over the years. Strong typing is going the way of the dodo. How often do you need to know what a pointer is any more? All the complicated things are getting hidden away in libraries or abstracted by the VM. There has never been a easier time to become a programmer than today.

I don't believe that everyone has to become a "programmer" any more than everyone has to become a "writer". But everyone should be able to write; a sentence, a paragraph, an essay, a script that their computer executes.

Exercising Git

2008-10-14T23:24:00.004-04:00

At work I use SVN for source control, but I've been itching to try out Git. I also recently stumbled across Project Euler, a series of math problems intended to be solved by writing small programs. I think you see where this is going. So - as of a few minutes ago - my Project Euler solutions (such as they are) are up on my github account.

I am a complete Git newb, so I followed some instructions on setting up Git and a new github repository. Since I can see all the code on github, I guess it's working.

My Project Euler solutions have been very fun so far. You're allowed to implement your solution in whatever language you want. I chose Ruby because it is the most expressive language I've found.

My solutions involving factoring primes have been a mixed bag. I think I did ok finding the largest prime factor of 600851475143 (problem 3), but my solution to problem 10 (sum all the primes below two million) was an ugly brute force. I really should have known a better algorithm (I won't say it by name here, but it was referenced in one of the Dark Tower books).

Some of the problems have been quite easy, but I'm only up to problem 17. I've peeked ahead and it looks like they get pretty tough. Sometimes it feels like using Ruby is cheating - for instance, problem 16 is "sum all the digits of 2^1000". Pity the poor C++ programmer!

The tricks I have up my sleeve are mainly inject, memoization and recursion. Recursion is especially helpful in simplifying problems. Memoization can be invaluable when you're performing expensive math. Inject is just plain fun - who wants to write all that boring loop code?

One reason that people might shy away from Ruby in Project Euler is the perceived performance benefit of more low-level languages. That is a complete red herring. When I was writing assembly language in high school, I read Michael Abrash's "Graphics Programming Black Book". One thing I learned was that extremely optimized assembly language will only net you a 2x to 10x improvement over compiled code. "Isn't that a lot?" you say? No - it is nothing compared to the gains you can usually get from improving your algorithms - often orders of magnitude. Using an expressive high-level language like Ruby allows you to concentrate on the high-level algorithms. I was especially proud of my solution to problem 15. I first broke the problem down with recursion. When that ran for more than a minute I realized I was solving the same problem more than once so I memoized it. Now it runs in less than a second even on my super-slow EeePC. Of course, there are much more elegant solutions out there - my solution that I was so proud of was far from the best. It just goes to show how far a good algorithm can take you.

This turned out to be a rambling post - sorry about that. In conclusion, Git and github are great, Project Euler is fun and algorithms matter. Good night!

Banish XSS Forever!

2008-08-28T22:18:00.000-04:00

The two biggest classes of web site vulnerabilities are SQL injection and Cross-Site Scripting (XSS). The ever-vigilant programmer can reduce the occurence of these problems, but proper architecture can eliminate them completely.

The SQL Problem Has Been Solved

I couldn't stop myself from referencing the XKCD SQL injection example. But for coders using prepared statements or parameterized SQL, SQL injection is already a bad memory. The problem of unsanitized data getting into your SQL can be avoided by following one rule: Never put any data into SQL. Instead of this:

$sql = "SELECT id FROM usertable WHERE username = '".$uname."';";
$result = pg_query($sql);

You do this:

$sql = 'SELECT id FROM usertable WHERE username = $1';
$result = pg_query_params($sql, array($uname));

Now when a script kiddie pastes some exploit code into my account sign-up page, he can't drop my tables or read all my password hashes. Little Bobby Tables can grow up to live a rich life without incurring the wrath of any more schools.

The XSS Problem is the Same Problem

SQL injection happens when unescaped data is interpreted by your SQL parser. XSS happens when unescaped data is interpreted by the client web browser. It's really the same problem, and the solution is pretty much the same.

I said "pretty much" because in the case of SQL, all the main relational databases have added APIs to allow "data" to be sent seperately from the "query". I don't see browser makers doing anything analogous. But you don't need your database to support parameterized queries natively - you can use a wrapper library that does it for you. Likewise, web site developers can adopt web frameworks that enforce the seperation of markup and text on the server side.

DOM View to the Rescue?

Stay with me here. The "traditional" way to build web pages programmatically was top-to-bottom. First you send your http headers, your DTD, your opening <html> and <head> tags, etc. until you finally send the closing </html> tag at the end. We're treating the web page as a string of text. Just because it is text on the wire doesn't mean we need to treat it that way. We should treat it as a tree, just like the browser on the other end does after it parses all that HTML.

Anyone who writes JavaScript is familiar with the Document Object Model (DOM) that is the web browser's internal representation of the web page. The page is a tree, with the <html> element at the root, and every element or chunk of text as a node. I propose that we write web frameworks that build web pages as trees - never having the programmer output any markup.

This system would start out with a basic empty web page.

html
|-head
| \-title
\-body

Set the title (please don't mind the syntax of my psuedo-code):

head.title.appendtext("Hello World")

html
|-head
| \-title
|   \-textnode value="Hello World"
\-body

Add a footer

body.appenddiv(class="footer").appendtext("© 2008 Me")

html
|-head
| \-title
|   \-textnode value="Hello World"
\-body
  \-div class="footer"
    \-textnode value="© 2008 Me"

Add our main content:

maindiv = body.prependdiv(class="maincontent")
maindiv.appendh1.appendtext("Hello World!")

html
|-head
| \-title
|   \-textnode value="Hello World"
\-body
  |-div class="maincontent"
  | \-h1
  |   \-textnode value="Hello World!"
  \-div class="footer"
    \-textnode value="© 2008 Me"

And so on. When you're done, your framework generates the HTML or XHTML or what-have-you from the tree and spits it out to the client. No markup is ever generated by hand and text can only go into text nodes or attribute values. The markup generator will automatically make the "©" for the copyright symbol and knows how to escape quotes embedded in attribute values as well.

This is the solution I propose - not the sisyphean task of "remember to escape your strings". Maybe I'll build it.

Stack Overflow

2008-08-28T11:12:00.002-04:00

I have been having so much fun with the beta of Stack Overflow - a new programmer help site by Jeff Atwood and Joel Spolsky.

If you're interested in the beta, you can get in here (scroll down a bit). Hopefully it will be live to the world soon. I expect scaling issues (not that it's had any yet, but I expect massive popularity).

In Praise of jQuery

2008-07-30T13:56:00.000-04:00

At work I'm starting to weave jQuery into the interface to our job database. Until now my Javascript needs have been limited to a menu at the top of the screen. Now I am trying to create a time entry interface, and it is complicated.

Each employee needs to be able to say "I worked on task alpha from 8:00 to 10:10, beta from 10:10 to 12:00 and gamma from 13:00 to 17:00". It needs to be easy to use, give them a list of the tasks that they are working on to choose from and make sure that their times don't overlap. This is all to say that some more advanced scripting is called for. I hacked around with vanilla Javascript for a while, but soon decided to check out one of the free libraries available.

Enter jQuery, and I could not be happier. The core feature of jQuery is that you can select elements in your document by xpath or CSS-style selectors. This alone is well worth the price of entry. For instance, originally I had to do a "document.getElementById("timebar")", use ".getElementsByTagName("div")" on that, and then check each element's class as I looped through them to remove a class. Now I just write: "$("#timebar .selentry").removeClass("selentry")". That's an if statement, a loop and a lot of long DOM method names I didn't need.

jQuery gives you all sorts of additional nice things - most of which I haven't tried yet. It includes:

Easy creation and insertion of new elements into your document tree.
Add, modify and remove attributes.
Modify styles on an element without wiping out unrelated styles.
Easily add event handlers with cross-browser problems smoothed out for you.
Iterators.
Handle ajax requests easily.
Plugins to handle even more things. (I was able to easily drop the date picker into my existing date fields.)

I resisted using a Javascript library for a long time. I thought they were just for fancy ajax and slide-in effects. I also worried that they would force all my scripts to work inside their framework. With jQuery at least, my fears were unfounded. Even if you just use it to find elements in your document tree it will be extremely helpful.

Bug Zero

2008-03-25T14:23:00.002-04:00

Bug zero is that the program does not exist. Fix that bug first.

The Legacy of Microsoft Office

2008-03-25T11:35:00.004-04:00

I'm a little bit late to this particular dance, but I feel like I have to say something about it.

Last month, Microsoft released the specs for its old binary Microsoft Office file formats. Lots of people thought this would usher in a new age of competing office suites. It will not. Microsoft Office binary file formats are monumentally, hopelessly complex.

Joel (on Software) Spolsky is a veteran of the Excel team, and he wrote a wonderfully insightful post about why this is so. He writes:

A normal programmer would conclude that Office's binary file formats:
are deliberately obfuscated
are the product of a demented Borg mind
were created by insanely bad programmers
and are impossible to read or create correctly.
You'd be wrong on all four counts.

The crux of Joel's point is that the formats were: designed around hardware constraints two decades ago; grown to account for every new feature since; all while maintaining 100% backward compatibility.

Joel is absolutely right about how the file formats got that way, of course. That doesn't change the fact that they are absolutely terrible file formats today. The startling thing is that Microsoft's new XML-based file formats are really just as bad. Microsoft has gone through a lot of trouble to translate all the data structures into new XML equivalents, but they still have all of the cruft of the old formats. This is not a system that can continue indefinitely.

All software with a long enough life faces this demon eventually. Backwards compatibility is a feature. It is an asset to software users who have old data. As with any feature, it is a trade-off, and comes at a price. That price in Microsoft Office passed the "too high" mark for me a long time ago. Likewise with Windows. I am more willing than most users to move to different systems, but everyone has a limit.

The greater lesson is that the cost of backwards compatibility should be carefully considered. In the end, extremely backwards-compatible software is just... extremely backwards software.

A Dramatic Reversal

2008-03-03T22:44:00.003-05:00

Microsoft's plan with IE8 was to make it use IE7's less-correct rendering by default, unless developers asked for "IE8 mode" explicitly. This was bad. They have now reversed their position. Money quote:

Now, IE8 will show pages requesting “Standards” mode in IE8’s Standards mode. Developers who want their pages shown using IE8’s “IE7 Standards mode” will need to request that explicitly

Maintainting Heirarchies in SQL

2008-02-13T20:37:00.001-05:00

I've refined my thinking a little about the optimal way to keep track of tasks in a relational database. Here is the newer version:

CREATE TABLE task (
 id BIGSERIAL PRIMARY KEY,
 parentid BIGINT REFERENCES task (id),
 taskname varchar(50) NOT NULL
 CONSTRAINT name_not_empty
 CHECK (taskname <> '')
);
CREATE TABLE leaftask (
 id BIGINT PRIMARY KEY REFERENCES task (id),
 weight INT4 NOT NULL DEFAULT 1000,
 starttime TIMESTAMPTZ,
 endtime TIMESTAMPTZ,
 CONSTRAINT startbeforefinish
 CHECK ((starttime < endtime) or (endtime IS NULL))
);

I want my tasks to follow certain rules:

All sub-tasks of a given parent task should add up to 100% of the parent task.
A task can never have exactly one sub-task because - according to the rule above - the subtask would by synonymous with its parent.
All tasks that have no descendants - and none that don't - should have a corresponding "leaftask" record.
All tasks must have a final root that has no parent - i.e. no looping back on your descendants.

I want to build structures into the database that will prevent invalid states (as defined by the above rules) from occurring. Today I am going to tackle the problem of a task descending from itself. To prevent that case, I need what in PostgreSQL is called a trigger. I'm getting ahead of myself though. First I need a way to list the descendants of a given task. Since I want it to happen all in the database, I need a stored function.

Stored functions in PostgreSQL can be written in many languages. I'm going to write mine in PL/pgSQL, a sort of SQL-related procedural language, because it is included by default with PostgreSQL. (PL/pgSQL is intentionally similar to Oracle's PL/SQL, if you're an Oracle person.)

CREATE OR REPLACE FUNCTION taskdescendants(parenttaskid bigint, OUT decid bigint)
 RETURNS SETOF bigint AS $fun$
BEGIN
 FOR decid IN SELECT id FROM task WHERE parentid = parenttaskid LOOP
  RETURN NEXT;
  FOR decid IN SELECT * FROM taskdescendants(decid) LOOP
   RETURN NEXT;
  END LOOP;
 END LOOP;
 RETURN;
END;
$fun$ LANGUAGE plpgsql;

This function will return the ids (each as a separate row) of any descendants a given task might have. I can use it in my SQL statements as if it were a table. For example, this will select all tasks descending from task 57:

SELECT decid, taskname FROM taskdescendants(57) INNER JOIN task ON decid = id;

That's all well and good for a generic function, but trigger functions have slightly different requirements. This will check that the new parentid doesn't show up in the list of descendants:

CREATE OR REPLACE FUNCTION taskrecursioncheck() RETURNS trigger AS $fun$
BEGIN
 IF NEW.parentid IN (SELECT decid FROM taskdescendants(NEW.id)) THEN
  RAISE EXCEPTION 'task cannot descend from one of its own descendants';
 END IF;
 IF NEW.parentid = NEW.id THEN
  RAISE EXCEPTION 'task cannot descend from itself';
 END IF;
 RETURN NEW;
END;
$fun$ LANGUAGE plpgsql;

CREATE TRIGGER parent_trap BEFORE UPDATE ON task
 FOR EACH ROW EXECUTE PROCEDURE taskrecursioncheck();

There - now it raises an error on invalid relationships and I got a handy function that my front end can call. Later I will have to figure out some way to ensure my other rules are followed.

Amazon or... Amazing?

2007-11-30T21:40:00.000-05:00

I am completely blown away by Amazon Web Services. S3 is wonderful in its simplicity and EC2 is revolutionary.

S3 stands for Simple Storage Service, and it lives up to its name. Put stuff up there and take it back down again. You can also have it generate auto-expiring URLs that others can access. Pay based on upload amount, download amount and storage amount.

EC2 stands for Elastic Compute Cloud. It is just a pay-by-the-hour server service. You provide the Linux image you want to run (or use one of their pre-built ones) and tell them to fire it up. The service assigns an IP address to the newly created machine and the rest is up to you. Pricing is such that it could destroy the dedicated server hosting business with a couple of small changes (basically a load balancer with a static IP would do it). It's biggest strength now is on-demand number crunching. Great for development/demo servers too.

Creating a Tree-based Task List in a Relational Database

2007-11-11T10:49:00.002-05:00

Things we want:

tasks
subtasks
task owners (responsible for that task)
task start time
task end time
prerequisite tasks

How do we represent these things in a relational database? Tasks are represented as a tree. Only leaf tasks will have owners and start/end times. This is because any non-leaf task will be made up entirely of its subtasks. In other words, when all the subtasks are done, the parent task is done. The start time of a non-leaf task is the earliest start time from among its descendants. Likewise, its end time equals the last end time from among its descendants (null if any are not yet finished).

Note: "bigserial" and "serial" are PostgreSQL abstractions for 8-byte and 4-byte integers that have an auto-incrementing default. They're mostly used for primary keys.

CREATE TABLE task (
    taskid BIGSERIAL NOT NULL PRIMARY KEY,
    parentid BIGINT REFERENCES task(taskid),
    jobid INTEGER NOT NULL REFERENCES job(jobid),
    taskname VARCHAR(200) NOT NULL
);

CREATE TABLE taskleaf (
    taskid BIGINT NOT NULL PRIMARY KEY
      REFERENCES task(taskid),
    employeeid INTEGER NOT NULL
      REFERENCES employee(employeeid),
    starttime TIMESTAMPTZ,
    endtime TIMESTAMPTZ
);

Problems:

Every task has a "jobid" - only the root task really needs this. Make the jobs themselves the root tasks? That wouldn't work in my pre-existing job scheme. Also, if performance was database-bound in my application, I could shift the tree-decoding work to the web server by selecting all the tasks by jobid.

Prevent circular references? I could make a trigger or rule in the database that would check the path back to a root node before allowing a parentid to be assigned. While I am at it, I can check to make sure the parent's "jobid" matches.

I looked at the "adjacency list" method, but I don't think it would work well for my needs.

Are You Watching These?

2007-11-08T17:53:00.000-05:00

Every time I watch one of the Google techtalks, I am blown away. As far as I can tell, this is how these things get made:

Google chooses a subject related to their business.
They find someone who knows that subject to give a talk.
They distribute the video so employees who couldn't attend can still get the info.

This is especially good because:

All subjects are related to their business.
They always seem to get the foremost authority on any subject (I bet they would reanimate Lincoln if they had a techtalk on the Civil War).
They give the video out to everybody.

PHP is Funny

2007-11-06T21:19:00.001-05:00

There are a lot of things that I like about PHP. For instance, the addition operator (+) is different from the concatenation operator (.). This prevents the common ambiguity in weakly-typed languages of "'1' + 1".

PHP has a lot of quirks though - due to its highly "organic" development. Variable variable names are a prime example:

$a = 'hello';
$$a = 'world';
echo "$a $hello"; // prints "hello world"

Confusing, no? Also, the object-oriented stuff seems tacked on (probably because it was tacked on) and the function to escape text so that it is HTML-safe is called "htmlspecialchars()". For a language whose main function is to output web pages, couldn't they have thought up a shorter function name?

Intro

2007-11-06T17:00:00.000-05:00

This will be my space to dump my software development thoughts. Welcome to a piece of my mind.