OpenPanel Development Blog

When you are designing a project with complex storage requirements and some demands on reliability and performance, a few options come to mind.

Even though at OpenPanel we have strong feelings about NIH, we didn’t think writing our own database store was the way to go. Many smart people have already written many different database backends, which means that in both quantity and quality the database area is well-covered. So, writing our own was right out.

The second option that comes to mind is to use the most basic kind of database available: a key/value store like Berkeley DB or GDBM. Combining a few key/value stores together yields a lot of flexibility, but there’s a lot of glue you need to write, then. In programming language terms, the key/value API is not powerful.

The only way out then seems to be SQL. The common choice seems to be MySQL, and with good reason - it’s robust, fast, flexible (supporting most of the SQL standard), comes pre-packaged for any distribution, and just about any sysadmin or PHP-developer you run into knows his way around it more or less. A close second would be Postgres, less popular with the common PHP developer crowd, more popular with seasoned developers in other languages, and understandably so.

However, we did not go for the path well-traveled. After evaluating our requirements, we realized we barely needed the power of a client/server model database with high concurrency support. Also, we figured choosing a slightly less popular implementation would deter people from messing with the database by hand.

We chose SQLite. It’s extremely lightweight, reliable, robust and surprisingly fast. Choosing SQLite means our database is just a file (directory, to be honest) in /var/opencore, where users and admins can’t run into it by accident while mucking about in phpMyAdmin - but when they really need to mess with the database, they can, with their familiar SQL idioms.

SQLite has most the features a developer would expect from a database; transactions, subqueries, decent indexing support, triggers, and room for extension with user defined functions. The only thing sorely missing is foreign key support, but that’s easily implemented as a bunch of triggers.

Even though we now had this powerful database engine, we put in a lot of effort mapping our idea of an object revision model onto it (more about that in a later post) - but SQLite made it a lot less painful.

SQLite’s only real limitation seems its lack of concurrency, which is made worse by the locking model used that seems to invite polling for access instead of forming some kind of queue. For this reason, OpenPanel keeps just one handle to the database file and manages exclusive and shared access to it via the normal pthread mechanisms.

Incidentally, the current (unpublished) version of our automated web application installer tries to shoehorn its data into a key/value store and it’s hurting - even from Python! We’ll probably rewrite those bits to use SQLite as well (but, of course, separate from the OpenPanel database).

Summarizing, if you are looking for a data store for your software project, consider SQLite. It’s as lightweight as most key/value store libraries but throws in a hell of a lot more featurewise.

Bookmark It

People requested a screencast of our command line interface in action as well, so here it is:

We’ll be adding a dedicated screenshots section to our website soon.

Bookmark It

The premise sounds easy enough: There’s no arguing with free. In the server control panel market we found a niche dominated by commercial players with a stranglehold on price and a bunch of products that annoy system administrators to the bone. So we set out to create something appealing — software with disruptive potential. With a product in our hands that ticks all the checkboxes in terms of nerd appeal, you would say that this would be as easy a sell as free blowjobs at a LAN party. Unfortunately, things are not quite that simple.

We went through the hoops submitting the news of our release to a dozen of the usual suspects, got a couple of hits and the first testers have arrived. There’s even a couple of people around doing more involved things like trying to port packages to other distros. It’s not bad and we appreciate all the people helping us out by playing with the beta, but a stampede it is not. Building a community takes time.

In terms of strategy, this may be the real reason for the ‘release early, release often’ mantra. Especially when your project is in a niche area, expect community building to take up maybe just as much time as actual development. We wanted to avoid getting distracted by actual users too early in the game, so our gut feeling at the time was ‘early, schmearly’. We had a couple of other reasons, some of them even sounded real good — like how there are too many open source projects in the field that need so much polishing that the market has been mostly ignoring them. But the fact of the matter is, all the hours of community building we’ve been avoiding up to the beta release will have to be put in double now. We’ll need to go from our current mode of reclusive introversion to full-blown extravert. Tough job for a bunch of nerds.

Bookmark It

It all started as an enormous itch that needed serious scratching. We’ve been dealing, in our various capacities at hosting providers, with many incarnations of commercial server control panels like Ensim, Plesk and cPanel. Not only were these, in our view, haphazardly constructed kludges of magic that actively worked against the operating systems they were installed on; we kept coming back to the fact that people were actually paying for such software. Through the nose, no less. This felt wrong and paved the way for the OpenPanel project.

We spent over a year creating something with the capacity to change the landscape, by combining the proper amount of openness and flexibility with a state of the art user interface and a license that should make trumpets sound from the heavens. People we’ve shown it to have been raving. But, now that we’re at the point to hit the submit button for all this on sites like freshmeat, all those little doubts start setting in. It’s scary as hell.

I’m convinced we’ve taken the right step by not fully adopting the ‘release early’ mantra — it has given us the chance to quickly reshape the design where needed (Heck, we’re on the second incarnation of the GUI). But this is the downside. If you release early, expectations are lower and you never get this neckhair-raising fear of throwing your work into the world. Alas.

We’ve opened up the 0.9 (beta) branch of OpenPanel as of today. There’s no longer a requirement to email us to get access, just follow the download instructions from the main site. Messages have also gone out to the usual places. The future is now.

Bookmark It