Acropolis { Website Management Software }

David Hofstadter’s law of programming:

It always takes longer than you expect, even when you take into account Hofstadter’s law.

This was in one of the closing chapters of Dreaming in Code, which I just finished last night.  I read the quote over and over again, and just thought it was wonderful, because it describes exactly my dilemma with estimating how long it will take to complete virtually any programming project.

I come up with a number, and I know–from past experience–that it will certainly take longer than that.  So I double it, or maybe triple it.  And, wouldn’t you know it?  It still takes longer than that.

I only wish sometimes I could go back to the days when I didn’t know that things always take longer than I think, and they only took about twice the time I expected.

I have a confession to make.  A while back I wrote about some good progress Travis and I made on Acropolis.  Well, the truth is we weren’t working on Acropolis, exactly.  We were actually working on another project for a different client– however we have been building this other application with a mind towards the exact structure we expect Acropolis to have.

We haven’t been able to spend any real time on the new version of Acropolis itself, but everything we do now, we do it with Acropolis in mind.

Despite my objections, I have been forced to make modifications on the old version of Acropolis (considering it is in daily use over here) and even though I am writing in straight PHP5, my experience with CakePHP has influenced me greatly.  Before, whenever working on one “module” of software, I would generally start with one main script that would load whenever someone accessed the page.  Then I would throw in a javascript file that would contain all my JQuery controls.  And then I would have several small scripts that would be called by AJAX requests, updating this or that element of the page.  The problem always was twofold: 1) I would get lost in my structure (or lack thereof) and 2) I found I was constantly repeating myself.  I would write one element in HTML, and then repeat the same element in one of the smaller scripts, or even insert a duplicate into the DOM using javascript.  All this to give the “appearance” of seamlessness, but beneath the surface it was anything but.

This time my approach was different.  I was working on Product Options.  I started off by creating a separate file that contained my class, which was called “Option.”  The class began with a constructor function that checked the Querystring for default variables.  The class then contained several functions, such as “existing options”, “add,” “remove,” “edit,” which contained both the mySQL queries and the corresponding HTML for the functions.  In the first script then, product-0ptions.php, I included this file and then called the appropriate functions throughout the code.  Whereas before I would build a select list for the existing options, I simply called the function as so: $option->existing_options();

Whenever a change was made, the ajax request would call the very same file, except the parameters were passed in the querystring.  The most important of which was “action.”  So, to update my existing optiosn, after say, one was added or removed, I would call this url:

product-options-class.php?action=existing_options&product_ID=4&manufaturer_ID=7

The include file would then use an eval() statement to call the requested function and return the correct HTML.  In other words, the exact function that first generated the HTML was the same one used to generate the updated HTML once the change was made.  You can’t imagine how much easier this has made my life.

I know examples would be really helpful right now, but of course, due to Hofstadter’s law, I simply don’t have the time.

More to come!

I mentioned in a previous post that I have been reading a book called Dreaming in Code, which chronicles the attempt by a team of programmers and designers to hammer out a Personal Information Management software (PIM) that will compete with the big boys - namely Outlook/Exchange.  I am about 4/5ths completed with the book, but it is already clear to me the project doesn’t really pan out–not only because I had never heard of their software, Chandler, before I started reading this book, but also because I’ve already read about three years of it and it seems like they are still only making baby steps.

Now it has been seven years and there is still no version 1.0.  While one may point to various flaws in their design process, or the decision to build their application in a desktop, rather than web-based environement (understandable if you understand that the project began a couple of years before GMail came out) but one could also oversimplify the whole issue by saying they simply underestimated the project.  They did not meet their self-imposed deadline, and therefore did not stay within budget.

I think any programmer can relate to that.  I know I can, after spending all day yesterday trying to complete a project I figured only had about two hours left.  I was to implement a nifty one-page checkout on one of our client’s sites, Cajun Grocer, like I had done with Acropolis (as can be seen on Strollers Direct.)  Spencer and our client alike were convinced it was a “cut-and-paste” job.  After all, I had already done it on one platform, how hard could it possibly be to move it to another?

To anyone out there who is not a programmer, but manages programmers: There is no such thing as cut-and-paste.  Unless two systems are exactly identical, they should be presumed to be absolutely different.  I may be exaggerating slightly, but not as much as you think.  In some cases the apparent similarity of two systems can actually create more havoc than if they were nothing alike, because you would not enter the project with any (wrong) preconceived notions.  You would assume nothing, therefore you could understand clearly the task that lay ahead.

So basically  yesterday can be summed up as follows: It works, it breaks, it works, it breaks, it works, it breaks…

And I still don’t feel like we are anywhere near actually getting back to work on the refactoring of Acropolis.

Best of luck, OSAF!  May something come of your efforts yet.  May something come of all of our efforts.

Not much has been done on Acropolis the past couple of days and I am not sure much more will be done in the next few.  Clients have been eating up my time lately with one-page checkouts, image galleries, site transfers, proposals, and God-knows what else.  I began last week feeling determined to get these all out of the way, so that I can then address the final remaining issues on Strollers Direct, so that I can then fully immerse myself in building out Acropolis.  Right now however, it still feels a long way off.

So anyway, as I am continuing to have dreams about programming, I thought this comic was apropos.  Enjoy.

Yesterday, in extreme programming style, Travis and I worked as a team to lay out a lot of the structural framework of Acropolis. If the marriage of JQuery and CakePHP is our goal, we may not be at an official engagement yet, but the courtship has currently begun. A design pattern I had been working out in my head and sketching down on blank sheets of printer paper several 3 a.m.’s over the last few weeks has now begun to emerge in living form.

The goal is for the page to never have to reload. Why? Because that is how desktop applications work. It is what your average person is used to. It is what “real” software feels like. In order to facilitate this we have to create our own form of organization, our own pattern, so that we don’t confuse ourselves in a mess of “spaghetti-code,” where JQuery functions are intermittently and inconsistently interacting with server-side scripting. I have been down that road a few times by now, and after a certain point, if you don’t have a solid organizational scheme to keep everything in its place, you begin to repeat yourself. This means more code to maintain, more to remember, and more that has to be changed whenever you add a new feature. This is no good. Now Cake takes care of a lot of this, and if you’re content to work with the prototype libraries it comes “out of the box” with, then you’re good. But if you’re like me and you just can’t tear yourself away from the vast potentialities available in JQuery, you have to forge your own path.

(Incidently, the two CakePHP JQuery “helpers” out there don’t seem to really fit the bill. One just helps you with your naming conventions, and the other is, well, not quite as “helpful” as JQuery tends to be on its own.)

But the standard naming conventions and organization of CakePHP just begs to be taken advantage of. It is so hard to write about this in the abstract, so let me just try to hit on a few key points. Hopefully I can elaborate upon these points as we go.

Partial Layouts + AJAX = Never Having to Repeat Yourself.

Small sections of your code that may need to be updated on-the-fly can be stored in “elements.” For our products module, then, we created a “row” element. When we list our products in a table, one product per row, we make use of this row element to format the data in the view. Each row in the HTML (<tr>) is connected to the actual “row” in the database by giving the HTML row an id parameter that corresponds with the Product.id. Whenever the information in that row (database) changes, the row (HTML) is updated via JQuery by calling a function in the controller that calls the same element (row.ctp) used to generate the row in the first place. (I feel like I could have said that in 1/4 the number of words but oh well.)

Predictable naming schemes mean one JQuery function can serve many functions.  Kind of like a crossing guard for code.

We constructed our own pattern for “actions” that can be used across modules (e.g. Products, Cateogories, Orders, etc.). We again made use of parameters inherent in HTML. For example, we use a link to fire the action to edit a product row. The code looks something like this:

<a href=”/products/edit/1″ class=”action” id=”Product/products/edit/1″>test</a>

What this basically means is that, first off, if your user is not using javascript the link will work normally, taking the user to /products/edit/1 where he/she can edit product with an id of 1.

If your user’s browser does use javascript, however, then JQuery takes over and things get very exciting. JQuery, upon document load, binds a click event to all elements with a class of “action.” When an action is clicked, JQuery takes the id of the clicked element and splits it up by ‘/’ and loads it into an array data[]. At this point it has all the information it needs.

var modelName = data[0];
var controllerName = data[1];
var actionName = data[2];
var actionId = data[3];

Now we are able to build an AJAX request to pull up a form and place it in a lightbox or whatever. In our case we chose to simply hide the table and then, after save, get rid of the form and re-display the table. Before we do that, however, we are able to update the row that has changed, which is easy, because we have the id of the row to be changed, and row.ctp is easily accessible in our elements directory. All we really need is a small function in the controller that delivers up the HTML of the one row based upon the supplied id.

I am sure it is difficult to picture, so I hope we will soon be able to post a link to a demo. But the result is a very fluid, and lightning-fast user experience that feels much more like “real” software than most people are used to experiencing in a browser. But what makes me excited is how squeaky clean and extensible this structure is. We have a few kinks to work out but once thats done our work will mostly involve customizing forms and the layout.

More to come soon…

I just have to keep telling myself that.

Programming, as life, has its ups and downs.  Today was most certainly filled with more downs than ups.  I am used to programs I write breaking for no reason, even though they were working fine yesterday.  I am used to neverending swarms of bugs.  I am used to poring over a problem for a straight hour only to realize I just spelled something wrong.  But I can generally take all this in stride because at least I never question my own competence.  Well, today–or more accurately, tonight, that confidence I usually take for granted has been slipping from me.

We are building Acropolis on CakePHP, as I have probably mentioned.  Now, although I have been using PHP for almost two years now, Cake is quite a bit different.  And it is even more different from PHP4, which is what I have mostly used, than PHP5, which I have only recently begun to tinker with.  The difference is between procedural verses object oriented programming (remember Alan Kay from my previous post) and perhaps one day I’ll feel like trying to explain the differences between the two approaches, but for now, take my word for it: the differences are huge.

Well, so I stayed a bit later at work today to try and hammer out one of the issues Travis was having trouble with.  Namely, we want to be able to list a table of information–say, products.  We want to be able to click on a product and get a dialog box, or “light box,” with fields for the product we want to edit.  We then want to be able to save the form, and then instantly update the information in the row we just updated, without the page ever reloading.

I know how to do this, alright?  I just haven’t done it in CakePHP before and I don’t want to just plow through it.  I want to work carefully within the beautiful structure Cake provides.  I don’t want to do it the way I have done before, which is to write out the table, and then when the update is made, execute a tiny script via AJAX that outputs the HTML for that row, and then do a replace.  Because the fact that the code for the row’s rendering would then be in two places, rather than one, just means more to keep up with and more hassle when I want to make changes.  And its just not elegant.

Well, I went to the #cakePHP chatroom for help, and I knew this was a big mistake.  Because as far as I can tell, there is a minimum level of competence you are required to have unless you expect to be ridiculed.  Of course I knew this, but I thought I was competent enough to ask a question like this.  But, I was wrong.  I am not sure if anyone ever really understood my question, since the only real answer I got was “Javascript.”  I was also told quite a lot of other things I already knew, and then your usual RTFM.  Well, ok.  I guess the truth is that I should have gone to the manual first, but I still don’t understand why the atmosphere in those chatrooms tends to be so hostile.  We are all at different places, alright?  And I still have to recall that one of the friendliest responses I ever got in the #jquery IRC channel, which I used to frequent on a daily basis, was on my first day using the library and the assistance was offered by none other than John Resig himself (the guy who invented JQuery.)  So I think that just goes to show you that a sign of greatness is not giving in to the urge to make others feel smaller.

I made some progress on this task, but I’m still pretty stumped right now and I know I’m just not in the right frame of mind just now.  Give me first thing in the morning and a cup of coffee and I’ll probably have it all worked out in less than half an hour.

Gone to get a beer…

I have been reading a fascinating book lately called Dreaming in Code, by Scott Rosenberg, co-founder of Salon.COM. He wites about Alan Kay, who single-handedly invented the concept of object-oriented programming. He compares the short history of software development to the longer history of architecture. In relative terms, Kay supposes, we are something like in the ancient times of computer programming. We pile thousands of lines of code, one layer upon the other, the way Egyptians laid heavy stones, one upon the other, in the construction of pyramids. Hundreds of years later, engineers discovered the “arch,” and were able to build huge cathedrals with considerably less material. This was the beginning of “architecture,” “literally the designing and building of successful arches. Kay’s proposal is to search for the “arch(es)” of software development–the simple, yet incredibly strong and scalabe structures upon which maginificent things may be systematically built.

I am not sure I interpreted his analogy quite correctly, but it nonetheless appeals to me.

This time around, in the development of Acropolis, I hope to spend more energy in the construction of a clear, flexible and extensible structure upon which a larger software application could be built- rather than, say, rushing to throw together as many functionalities as I can by our proposed deadline- no matter how many bugs. The skilled designers at 37 Signals say, “don’t build a half-assed product. build half a product.” It doesn’t sound encouraging to a lot of product managers, but there is something so honest about that that has a far, broader appeal. If you face a tough deadline, it may be the deadline itself that is at fault–not your abilities, not your environment, not your boss. Or, on the other hand, it may be that you are planning to build more than you should.

So far, we have spent more time thinking through the coding and visual structure of Acropolis than we have actual programming. Instead of trying to check off as many “features” as conceivably possible, we are trying to find the common ground between all of these features, so that we may clearly see what work we can do for all of them at once, and where the blanks will remain.

I was intending to write about the basic structure of the Acropolis application, for whatever readership we may have acquired in the last couple of days. But now, I see I do not have nearly the energy to do more than what I have just done, which is to introduce it.

More to come…

And no, I’m not talking about money.

The concept of a web application has been around for a while, but it has not been until recently that programmers, working with designers, have been able to produce apps that look anything like what your average user thinks of when hearing that word.  The kind of application most people are used to is something that runs off your desktop, like Microsoft Word or Excel.  Applications have a clean interface, and a familiar structure that remains intact as one moves through the many actions available.  And, well, the page never reloads.  It just changes as your requests change.  And there is always a consistent structure that never changes, such as the File, Edit, View… menus at the top, or other toolbars.  The very browser you are using is just one such example.

Other than web email, the first Internet application I remember using was the shopping cart software the company I worked for in 2003 had developed.  There was a list of “managers” on the left hand side of the screen, things like Product Manager or Category Manager.  If you clicked Category Manager, for example, you were shown a dropdown menu that had a list of your categories.  You could then burrow into the categories by clicking “View Subcategories” or “Go up a level.”  I could not have imagined then how clunky this kind of interface would eventually seem to me, but it did get the job done.

One of my first moves, however, when I had the opportunity to create my own shopping cart software was to replace the dropdown menu concept with something more visual–the expandable/collapsible “tree.”  This is similar to the directory tree structure finds in many Windows-type application and it is used to retrieve files.  One does not need to reload the page several times to find the category he/she is looking for.  Simply click the plus/minus signs to open/close “nodes” or “branches” of the tree, click on the category you want, and voila!  You’re there.

One will still find this legacy in the upcoming release of Acropolis, but we are taking it quite a bit further.  I want the entire application to be “rich,” in the sense of “Rich Internet Applications” (RIAs).  RIA’s are web based, and are said to be bridging the gap between desktop applications and web applications.  The page never, or hardly ever reloads.  New information is loaded in via AJAX.  Elements on the page slide up or down, fade in or out, disappear and reappear with the click of a button.

The only problem is, applications like this have generally been much more difficult to build than the old-school app I used on my former employers sites.  And consequently, they are more prone to bugs.  The problem becomes one of organization.  If you can’t break your application down into pages, you have to find another model.  And you have to account for different types of elements.  You need a framework for your application.  It is looking like Cake PHP will be just the ticket for us, in terms of keeping our application well ordered so that JQuery (the framework used for AJAX and other dynamic page changes) can work its magic with ease.

Last night I was thinking about the problem of “memory” in the Acropolis application, and came up with the following requirements:

• When moving between tabs, one should always be able to continue where one left off in previously selected tabs.  You shouldn’t have to keep starting from the beginning every time you click the “Products” tab.
• Any items you have “selected” in a given tab should stay selected until: A) you have unselected them, or B) some action has been performed upon them.
• If you have moved through multiple pages, you should always come back to the page you were just on after performing an action, such as adding a new item or editing an existing one.
• All of ones search parameters (search phrase, category, manufacturer) should stay active until one deselects one of them, and moving “backwards” through a breadcrumb should be easy to follow.

More to come soon!

I have begun this blog to follow the development of a Web-based software application code-named Acropolis.  (Actually, I just wanted to say “code-named” because it sounds cool.  That is more than likely going to be its final name as well.)  I like to describe Acropolis as a WMS, which stands for Website Management System.  The reason is because I have used various beta versions of the software to manage brochure-type websites, as well as e-commerce sites.  It has a blog component and a calendar component, in addition to the ability to display and sell products.  So I suppose my hope is that this will be more than yet-another-shopping-cart-software.  But if in the end it just ends up being a clean and flexible e-commerce application that helps myself and a few others run successful online businesses, that will be good enough for me.

A Little Background

I began writing Acropolis one afternoon in March 2006 when I decided I wanted to build an e-commerce website for my fiance and I, which eventually grew into Organic Fair Trade.  At the time I was the full-time developer for an online lighting retail store which had its own software it had built in-house using classic ASP.  There was a brief moment when I considered simply “borrowing” that software to create our site, but I decided not to for two simple reasons: 1) I didn’t want to get sued, and one of my bosses was very much the kind of person who would consider doing just that; and 2) I didn’t like the software that much, and I felt I could do much better.

In order to alleviate any possible future suspicion that I had in fact borrowed or adapted the software to my needs, which was proprietary and belonged to the company, I decided to choose an entirely other language, which I had never used before: PHP.  It took me surprisingly little time to create most of the basic functionalities I needed, and within a month I had our site up and running with a few products and a simple shopping cart that connected with PayPal.

The months passed and I continued adding more features to the software and, consequently, our website.  One day I was approached by a friend of a friend who had asked if I could implement his currently static site into some sort of e-commerce software.  I charged him $1,500 and set him up with what I had decided to call Acropolis and imported thousands of products into the database.  I also developed several new functionalities to suit his needs.  That site is Red Hill Supply and while it may not look like much, it is very much pulling in the mulah.

Early the next year I was referred by the owner of Red Hill Supply to do the same thing for a friend of his, who I also charged $1,500.  This site was JMS Online.  And again, for this client, I implemented many new functionalities that made Acropolis even better, and transferred what I learned over to Organic Fair Trade.

By March 2007 I decided I had had enough with my overbearing boss at the lighting company who would just as soon sue me as look at me, and I took a job with Comit Technologies in Lafayette, Louisiana.  During my interview I demo’ed Acropolis to the owner, Spencer Hoyt, and tried to sell him on investing in its further development.  He didn’t bite, at first, but he was definitely interested and I was hired as the company’s first programmer.  After a few months he made the decision to build all in-house websites on Acropolis, which has most recently manifested with Strollers Direct.  This site is built upon the very same lineage of software that began so humbly less than two years ago, but it now does things I could not have even dreamed of at that time.

That was then, this is now…

We have recently added two very talented members to the Comit team-Travis Boudreaux and Josh Chauvin-and it seems that the three of us are going to be the core team for an ambitious and exciting undertaking regarding this software I have come to know and love.  We are going to be redesigning the software from the ground up.  But redesigning sounds so daunting, I prefer the word refactoring.  Part of the problem with Acropolis is that I never imagined it would grow as big as it did, and so there were a number of organizational decisions I should have made in the beginning, but didn’t.  Another side of that is that I was working in a language I had never used before.  For better or for worse, those humble beginnings provided the shaky foundation that Acropolis has been built on, and the software has become too complex to rely upon it any longer.

From the programming side, Travis and I are going to be re-building Acropolis in the Cake PHP framework, which is basically PHP’s answer to Ruby on Rails.  In addition we are going to be closely integrating Cake with JQuery, an awesomely powerful Javascript library that allows for all manner of special effects and AJAX functionalities.  These two, what I see as being among the best server-side and client-side platforms available today, have the potential to become quite a formidable force when combined.  The interesting challenge here, however, is that there does not seem to be a consensus right now in the web development community about how these two can be best combined, and at this early stage, it is looking like we will need to forge our own path in that regard.  But more on that later…

From the design side, Josh is taking Travis’ and my sketches on the whiteboard and creating some pretty promising interface designs that make me both very excited, and very impatient.  To date, no designer has set his mouse to the look and feel of Acropolis, and while it has done very well in terms of its outward performance, the back-end administration tool, visually speaking, has always been less than inspiring.  I will be posting some mockups of Josh’s work in the near future.

And so the story begins…

But it won’t just be about programming and graphic design, because I think another part of this story that is just as important is the real world limitations we are working within in this small, web design company in south Louisiana.  “We still gotta pay the bills,” is one of Spencer’s favorite aphorisms, and to do that, we have lots of clients.  Sometimes I can hardly get any programming done, for Acropolis or for our clients, because my phone just doesn’t stop ringing.  My to-do list is a mile long right now, and already I have upset several people in the last week due to my lack of promptness, which I assure you, is not for a lack of trying.  But, we are committed to the redesign of Acropolis, and so I believe it is going to happen.  We are shooting for March 15th, on paper–Spencer wants it done before then–I think it will almost certainly take longer, but I am willing to give it my best shot.  With the right tools, you never can tell what you might be able to accomplish.

So, I will be trying to keep this blog updated daily with our latest coding puzzles as well as anecdotes about “life at Comit.”  Perhaps it will generate a little interest about what we are doing over here, but more importantly, I hope it helps all of us involved feel a little more excited about what we are doing–and perhaps a little more impatient as well.