| FeedSee | |
|
An Architect's View Web Feed An Architect's View As folks have no doubt realized, I'm using Clojure more and more these days and just embarked on a project at work to replace the CFML-based ORM "Reactor" in our codebase with a much more lightweight ORM written in Clojure (enclosed in a lightweight CFML wrapper). While I've been sketching out the code for that, I've been using the clojure.contrib.sql library, one of a fairly large selection of somewhat ad hoc utilities contributed over the last several years by various Clojure community members. As part of the 1.3.0 release of Clojure, currently in development, the Clojure/core team have decided an overhaul of the contrib libraries is needed and so they requested dedicated library maintainers to step forward to work on newer versions of popular libraries as they are promoted to new namespaces within the Clojure ecosystem. Once clojure.contrib.sql's new namespace was approved a the new clojure.java.jdbc github repository created, I started work on migrating Steve Gilardi's code, removing dependencies on old clojure.contrib libraries and getting a maven build working. Since then, I've put out a general call for feedback on the Clojure mailing list and started to make some changes to the library itself:
Generated keys are returned as a structmap from insert-record and as a sequence of structmaps from insert-records. Generated keys are also returned as a structmap from calls to insert-values that only perform one insert (insert-values can be used to insert multiple rows in a single SQL statement). 0.0.1-SNAPSHOT builds are already available from the Sonatype OSS snapshots repository. If you're using Leiningen, just add the following to your project.clj file:
:repositories { "sonatype" "https://oss.sonatype.org/content/repositories/snapshots" }
:dependencies [[org.clojure/java.jdbc "0.0.1-SNAPSHOT"] ...]
I've already received some good feedback and I'm starting to accumulate issues on the github site as I figure out what needs doing and how best to do it to take the library to the next level. If you have feedback, feel free to post it to that Google Groups thread, here in comments or directly to me (using the Contact Me! link on the right). As for the lightweight ORM, that's working pretty nicely. On top of the clojure.java.jdbc library it required just fifty line of (well-commented) Clojure code, a generic persistent CFC base class (about 100 lines) and a simple ORM service (25 lines) to create beans and inject the Clojure ORM code. Beans can load, save and delete themselves (using the injected Clojure code). If work allows me to open source it, I'll add it to my cfmljure project as part of the examples. My cf.Objective() 2011 Schedule - Now that the online scheduler is available, I've been picking the sessions I'll be attending at cf.Objective() 2011 Thursday, May 12, 2011
Friday, May 13, 2011
Saturday, May 14, 2011
Some difficult choices there because there are some killer sessions on the schedule! To UUID or not to UUID? - A discussion that crops up from time to time is whether primary keys should be UUIDs or simple auto-incrementing integers. There are pros and cons on both sides. UUIDs provide universal uniqueness which allows data to be stored across multiple databases, merged and so on. Auto-increment primary keys are easy to work with and efficient for indexing - but cause problems when you need to deal with data created across multiple databases, such as data migration and merging. UUIDs, as 36 character strings, take up a lot more space and can be much less efficient as an index. But what about storing UUIDs in a more efficient format? A DBA recently recommended using BINARY(16) since that holds 128-bit data and a UUID is usually decomposable as two 64-bit long integers. The question is how to do this? And, I guess, another question is: is it really that much more efficient than a 36 character string? When I first looked at implementing the DBA's advice, I couldn't find a type in cfqueryparam that would support BINARY(16). There's BLOB... and I guess that would work if I could provide a byte array? Then the question is how to get a UUID - a 36 character string of hex with '-' separators - into a byte array in the first place? And then you have to ask how efficient that would be (under the hood, the UUID generator will create two 64-bit longs and then have to convert that to a string before CFML gets it's hands on it!). Then another issue is the raw speed of UUID generation in the first place. Adobe made some big improvements between CF8 and CF9 here (100x faster) but generating UUIDs is traditionally still quite slow and in CFML they get converted to a delimited hex string so you can't even get at the underlying binary data. When I did some research, I found that the UUID generator built into Java (since Java 5) can only produce type 3 (name-based) and type 4 (random) UUIDs - it can't create time-based (type 1) or DCE security (type 2) UUIDs. There are third-party generators that can produce time-based UUIDs and which perform better than the built-in Java version. Overall tho' this sort of fiddling around is something better suited to a low-level language like Java. We want a raw UUID. We want to get the two 64-bit longs and write them to a 16-byte array and use that as our BINARY key. As an experiment, I tried Johann Burkard's lightweight time-based UUID generator - see below - and wrapped it up in Clojure to create the value I wanted:
(defn uuid [] (com.eaio.uuid.UUID.))
(defn uuid-as-byte-array []
(let [u (uuid)
^long lo (.getClockSeqAndNode u)
^long hi (.getTime u)]
(-> (java.nio.ByteBuffer/allocate 16)
(.putLong hi)
(.putLong lo)
(.array))))
There may well be more efficient ways of constructing the byte array, such as direct write operations and bit manipulation, but this seems clean and simple enough. I'd be very interested to hear what others have done around UUID and/or binary indexes on large tables and, in particular, how you've deal with that from CFML? See Johann Burkard's comparison of UUID generators for feature / performance analysis. See also Mark Kruger's recent post about leveraging Java UUID from ColdFusion - and the performance improvements possible (and read the comments). FW/1 Roadmap, DI/1 etc - Last year I'd talked about getting at least an alpha of FW/1 2.0 out the door by year end. It didn't happen. I was reminded of this on the FW/1 mailing list the other day and I provided a fairly detailed response on the plans for FW/1. I figured, since I haven't blogged about FW/1 in a while, I should post that roadmap response on my blog as well. Is the 1.x version outdated? No, the 1.x stream will continue to be supported. It will get bug fixes but it will not get new feature - except where I want to provide a migration path for people moving to 2.0 (for example, if I remove certain things marked deprecated in the docs, I'll provide the new API in a 1.x build to make it easier for folks to transition to 2.0). Expect at least a 1.3 version, possibly more. These versions will continue to support CFMX7+ as well as the older versions of Railo and OpenBD that 1.2 currently runs on. Wasn't 2.0 imminent late last year? Yes, FW/1 2.0 was very imminent at that point but work and then conference preparations overwhelmed me and it got put on the back burner for a while, along with DI/1. Since a respite is coming in my day job (World Singles) - we're launching the second phase of our new platform soon - and I have a couple of months before the next conference, I expect to get back on top of FW/1 2.0 soon and once an alpha of that is available, I'll work on DI/1 to get an alpha of that out as well. And then I'll probably focus on cfmljure for a while, as we start to add Clojure to our production code base at work. And then it'll be beta versions of FW/1 2.0 and DI/1 and cfmljure, round and round, until they're all three "baked" and ready for official releases. So DI/1 is coming? Yes, DI/1 is still on my radar - I have plans for it at work so it will definitely happen! It's just been delayed (ironically, by the pressures of work). Since FW/1 2.0, DI/1 1.0 and cfmljure 1.0 will all be used in my production code at work, you can expect them to receive regular updates once I actually get to the alpha releases! A Round-Up of Scotch on the Rocks - Following hot on the heels of OpenCF Summit in Dallas, I headed to England (this time with my wife) for Scotch on the Rocks in beautiful Edinburgh, Scotland. I spent a lot of time engaged in hallway talks during the day and long discussions in the bar during the evenings so I must confess that I saw very, very few of the sessions. I would have attended more sessions but I was generally a bit late to each one (due to aforementioned hallway conversations) and it seemed that every session I wanted to attended was already completely packed out the door - my sense is that the Apex International Hotel, while perfectly wonderful in many ways, doesn't have enough large conference rooms to handle the number of attendees that Scotch is attracting these days. With that complaint out of the way, how was the conference? Well organized, well attended and packed with great content. Adobe kicked off day one with a look at ColdFusion Builder 2 - very impressive! - and the news that the public beta is available on Adobe Labs. I've been impressed with ColdFusion Builder since launch - I bought my copy the day it was released (within minutes of release, in fact) and it's been open pretty much 24x7 on my main development machine ever since. I think it's a great productivity tool and well worth the $299 sticker price. When I saw Ram Kulkarni preview some of the new Builder 2 features at MAX, I drooled. This would make me even more productive! And now the beta is here, everyone can see the new features with massive improvements in performance, code assist, keyboard shortcut productivity and several major enhancements to the extensions functionality, this is definitely a major release and I'll be upgrading as soon as that's officially possible. Kudos to the Adobe engineering team, two of whom were at Scotch, for all their hard work on this great IDE! After a short homage to the departing product manager, Adam Lehman, he took the stage to provide some early information about ColdFusion X. Promising to be the "biggest release ever", Adam started out with some infrastructure changes: the long-overdue replacement of JRun with Tomcat, the long-overdue removal of Verity (now we have Solr / Lucene) and the long-overdue addition of Axis 2 (with Axis 1 likely to be removed in CF11). Next up, a complete overhaul of the scheduled task system. There were a few other things but one that caught my eye was the addition of closures, something I've wanted in CFML for several years (now all we need is a decent collection class library to use them with). I was a bit surprised by how muted the audience response seemed to be - perhaps the impact of these important changes won't be understood by most CFers until they see what other changes this enables in the product (swapping JRun for Tomcat should improve stability and performance, as well as opening up a whole new world of tooling and support and add-ons that Tomcat provides access to). I wanted to attend Ben Nadel's talk on regex but it was completely packed. I took in a bit of Peter Bell's talk on requirements and estimating but I've seen versions of it before (although it had some major new stuff in it this time around - and will be getting a complete overhaul before cf.Objective()!). Next up was my FW/1 talk and we had to start a bit late because the room was over-heated and over-full and Andy Allan brought glasses of water for everyone to help folks stay cool! The talk seemed to go well and, judging from the buzz on Twitter, was very well received - I'll blog about FW/1 and plans for new releases shortly! I missed a couple of sessions but really wanted to go to Rob Rawlins' "The Pen Is Mightier Than The Keyboard" but it was completely packed and I couldn't get in (and according to Twitter, it was great!). Somehow I missed the Adobe: Meet The Team session - more hallway conversations, I think - and then it was into the bar for a long night of networking and chit-chat (OK, and beer). Day two kicked off with a keynote from Mark Drew and Gert Franz that echoed many of the sentiments from the OpenCF Summit: be proud of CFML, get involved with open source - applications will help grow the CFML community, not utilities / frameworks. Gert followed that with a look at some of the innovations in Railo 3.3 and Railo 4.0 including distributed session storage, command line CFML, deep Java integration (some of which looks planned for ColdFusion X), closures (also in ColdFusion X), cfscript language="java" and so on. Judging from Twitter, it was well-received (I didn't actually attend - it was too full by the time I got there, again!). I wanted to attend John Whish's talk on ValidateThis but - guess what? - it was in the small conference room and completely packed so I couldn't get in. I gather than Ray Camden's jQuery Mobile talk was excellent, despite some early technical problems. I did, however, get into Andy Allan's "Home for 5pm" talk which was absolutely awesome! He talked about how much time and productivity we lose with task switching - we're in the middle of something, get interrupted, have to switch to some other issue and then waste time getting ourselves back to where we were. The solution, he proposed, is Mylyn and/or Tasktop. These Eclipse plugins keep track of "context", which files are open, what tickets you're working on, and so on, and make switching tasks pretty much automated. I had tried Mylyn before but didn't really "get it"... during Andy's talk I'd installed Mylyn on my netbook along with the Unfuddle plugin and was quickly able to switch between active tasks and have Eclipse automatically open and close all the relevant files for me! Amazing! For me, this talk alone would have been worth the price of entry, in terms of the productivity it's given me! That sort of ended the conference on a high for me - I got caught up in several long hallway conversations and missed the last two sessions! The evening was, of course, another long one with lots more networking, chit-chat and beer. The Apex International Hotel is a lovely venue with terrific views of the imposing castle, especially from the fifth floor restaurant (which has excellent food). The bar stays open nice and late (for hotel residents). Lots of options for excellent food (and drink) are within easy walking distance. My only complaint is the size of the conference rooms - something I hope the Fuzzy Orange crew will address for next year. The next day my wife and I took Ray Camden up around Loch Ness - using the quiet, mostly single track road that winds along the eastern edge of the Loch and where you can get right down to the surface of the water - beautiful! Here are some pictures I posted on Facebook (you don't need to be on Facebook to view them!) and you'll see four of Ray on the phone, telling his family back home that he's "actually standing on the edge of friggin' Loch Ness!!!" - it was a very cute moment! My wife and I went on to rack up 1,300 miles on our rental car visiting friends and family all over the UK, followed by a trip to Denver for a cat show, before getting home, whereupon I took to my bed, sick, for 36 hours... which is why these two conference write-ups have been a bit slow coming! A Round-Up of OpenCF Summit - The first annual OpenCF Summit was almost a month ago but I've hardly stopped traveling since and then I got sick! So, finally, here's my thoughts of how this inaugural event went... The overall impression was one of friendliness, openness and collaboration. From the moment I arrived at the hotel to register, I was immediately swept up in the hackfest where all attendees were able to contribute to an application to help a not-for-profit organize its volunteers. This application grew from a skeleton to a fairly full-featured application with a nice user interface over the course of the summit as various attendees worked on the application. It was great to see such team spirit among attendees, especially since the development environment - Mach II, Open BlueDragon, Google App Engine - was new to most of the contributors! The opening keynote, by Matt Woodward, on Monday challenged us to be proud that we write CFML, instead of almost apologizing for our favorite technology. Matt emphasized the power of our technology and the upbeat state of the union for CF developers, the growth in open source within our community - although tempered with a caveat that we can (and should) do better: Rails 3.0 was held up as an example of an open source project with thousands of contributors whereas we often have less than half a dozen on most CFML projects. Matt also made the important point that what we need to help CFML grow is great open source applications, rather than yet another utility / framework. The sessions all took place in one room, as a single track, which I think helped people get to know others at the conference and fostered more of that sense of community and collaboration. The sessions were varied, with some covering the business and/or cultural aspects of open source, some covering specific technical topics (scalability, REST, development environments) and others covering more esoteric topics (such as Denny's deep dive into the CFEclipse project or Ean Schuessler's fascinating look at how the Government 2.0 movement is embracing both open source and community contributions). I gave two talks focused on the open source ecosystem. The first was an updated version of my "Open Source Landscape" talk that looks at how open source projects and collaboration have evolved in the CF community. The second was a new talk that looked at how two "rising star" languages are evolving as open source projects - Scala and Clojure - and comparing that process to the remarkably similiar ones behind Railo and OpenBD. On the third day, the main track featured deeper looks at the two main open source CFML engines (OpenBD and Railo) whilst Unconference sessions went on in other rooms, featuring content requested by and created by the attendees themselves. Finally, we all got together again to review the conference and the progress of the hackfest application - and give prizes to the biggest contributors! As a sponsor, Railo Consulting was very happy with the event. We spoke to a lot of attendees about projects where we could be of assistance, so we expect to get several consulting engagements out of the summit. In addition, of course, a number of the attendees were not familiar with either open source engine so this was a good opportunity to show them what open source alternatives are available - several were already looking at other open source technologies (often for policy reasons, as well as cost), so they were glad to discover that they didn't have to abandon CFML in order to satisfy their needs. The facilities were excellent! The convention center was a good venue with wifi, plenty of power strips, coffee and nice boxed lunches (catered locally, I believe). The hotel was very nice and modern with spacious rooms and there were several places to eat and drink within easy walking distance. I'd consider the event to be a great success and I'm looking forward to next year! The Joy Of Clojure - The final "Early Access" version of Manning's "The Joy Of Clojure" by Michael Fogus and Chris Houser is now available with the print edition becoming available on March 25th. I bought this back at the beginning of September 2010 and it's been wonderful to watch the book evolve through the early access program. It's a very entertaining and, at times, challenging book that really gets into the "why" of functional programming and the fun you can have with a language in the Lisp family. I find myself re-reading chapters over and over again to really soak in the concepts and approaches - and functional programming is pretty much what I started with back in the early- to mid-80's. It's very rewarding! This final version now includes the foreword by Steve Yegge of Google (a very entertaining and thought-provoking blogger whom I've highlighted a couple of times here on this blog). I'd like to quote the closing paragraph of Steve's foreword: "Lisp - the notion of writing your code directly in tree form - is an idea that’s discovered time and again. People have tried all sorts of crazy alternatives, writing code in XML or in opaque binary formats or using cumbersome code generators. But their artificial Byzantine empires always fall into disrepair or crush themselves into collapse while Lisp, the road that wanders through time, remains simple, elegant, and pure." And that, to me, is why Lisp, Clojure and this book are such a "Joy" to work with! OpenCF Summit - My Slides Online - My two presentations from the OpenCF Summit are available on my presentations page: CFML and the Open Source Landscape (updated), Open Source Language Evolution (new). The conference team are putting up links to all the presentations as they are made available, via the conference schedule page. I'll blog my thoughts about the conference in a separate post shortly, possibly on the Railo Technologies blog. Passing the torch - By now, many of you will have seen Adam Lehman's blog post about changes in ColdFusion's product management. A few people have said this spells doom for ColdFusion. Already Adobe's Community Professionals and Community Champions have piled in on this and voiced their support for Adam - and decried all such concerns. Reality check folks: ColdFusion has had a number of Product Managers over the years and it's never been a cause for concern. Adam has been one of the most passionate champions for the product that we've seen in a long time and his efforts should be praised. When Adobe bought Macromedia, my role as architect was eliminated and one of the roles I discussed with management was ColdFusion Product Manager. The travel was the killer. I would have loved the job - just as Adam has loved the job, despite the drain on his personal life with all those trips to Bangalore. Let's be clear here: ColdFusion development went to India years ago and that team has done us all proud! CF8 and CF9 are two of the most significant releases in the history of ColdFusion. Hemant and his team are very passionate about what they do. We've seen massive improvements in the product over the last couple of releases and CF X should be another milestone release (even if it takes a "bit longer" than usual). The CFML community is a very tight knit and relatively small one. We should be supportive and inclusive. Let's not see infighting reduce the community. Whether you're an ACP / ACC or not, you're passionate about CFML and we all want to see it succeed. Let's all wish Adam (and Alison) success in their new roles and welcome Tridib and his team as the new shepherds of ColdFusion, just as we've welcomed various other product management folks in the past! Let's show them what the CFML community is all about! cf.Objective() 2011 - two new presentations - cf.Objective() has just posted the draft schedule for 2011 and, as always, it's packed full of great sessions. I'm very pleased to announce that I am speaking again this year and I'll be bringing two brand new presentations to the conference:
The former will draw on work I did at Broadchoice, building their SaaS-based CMS platform, and more recently at World Singles, building their new internet dating platform. It will cover some of the challenges such systems provide (and various ways to tackle them). Sacramento CFUG will get an early sneak peek of this talk in a few weeks. The latter will cover a style of programming that's been around for decades and has seen a massive resurgence lately as concurrency, parallelism and efficient use of multi-core / multi-CPU machines have become increasingly more important. While the talk will show examples from Clojure and Scala, I will also show how some of functional programming's powerful idioms can be used in CFML! OpenCF Summit - Even More Affordable! - The folks behind the first (and hopefully annual) OpenCF Summit have taken another look at their budget and, after working with a number of partners, have been able to reduce the price of the conference to just $30! Now the premier event about free open source CFML is almost free! What are you waiting for? Register for this ground-breaking event and learn about what free open source software can do for you (and what you can do to support that movement)! What do you want from Santa? Railo 3.2! - Happy Holidays! Whether you've been naughty or nice, you can read all about the new Railo 3.2 release with many community contributions and new core features! Michael Long posted a comment on my "Are Objects Bad?" post which I wasn't really sure how to answer effectively. Michael feels that if you're just working with functions, and grouping them so each file deals with specific concepts, you're starting down the road to OOP anyway so why not "bite the bullet" and use full-on OOP... There are two things that don't sit well with me about Michael's comment. The first is that OOP is much more about conflating state and behavior to create self-contained 'smart' entities that more closely model the real world. Grouping related functions together is something folks have been doing in pre-OOP languages for decades. But it's interesting that Michael would make that observation and I can see how such a grouping could be seen as a first baby step toward OOP. The second part of Michael's comment that made me itch a bit was "get the additional benefits it [OOP] provides". As I conceded in my original post, not everyone thinks OOP is all about benefits. Part of what made me pull back from 'pushing' OOP was that it has a number of downsides too. There's no 'right way' to design an OO application, which bothers a lot of people (true, there's no 'right way' to design a on-OO application either but many folks are either overwhelmed by the apparent choices or frustrated by the way that a poor OO design seems to punish you much more sometimes than a poor non-OO design). Another potential downside is the 'class explosion' problem and the ensuing navigation problems it can cause in understanding the code base. Other problems arise out of poor cohesion, excessive coupling and a whole slew of anti-patterns. Many of these problems aren't entirely the sole purview of OOP but they can often be magnified in an OO design - and they are real problems for people new to OO as they drown in classes and patterns (and anti-patterns). But there's a more insidious problem that OOP has slipped into our day-to-day lives that most of us really haven't noticed yet! Where we're beginning to see it in our web applications is issues of concurrency, where an object's state can be manipulated by more than one thread / request at the same time. Rather than try to explain it in more detail, I'll defer to Rich Hickey, creator of the Clojure language, who gave this excellent keynote talk at the JVM Languages Summit in 2009: Are We There Yet? - a talk about state, identity, value, time and other concepts. Call for Speakers - cf.Objective() and OpenCF Summit - Both cf.Objective() and OpenCF Summit have their call for speakers/proposals open at the moment. Bob Silverberg talks about the cf.Objective() CfP on his blog and you can read about the OpenCF Summit CfP on their blog. The important thing to note in both cases is that there are calls open: suggestions and proposals. The suggestions page is where you make suggestions about topics you'd like to see at the conference. The proposals page is where you submit yourself as a speaker and describe your proposed topic. Even if you've submitted a suggestion for a topic you'd like to present, you still need to submit a proposal in order to be considered as a speaker! You need to connect with either your Twitter or Facebook account in order to submit suggestions or proposals - welcome to the social media era! - and proposals are open until early January for both conferences. The OpenCF Summit blog has lots of background information on the "engage" application in use by both conferences, as well as doing a great job of explaining what the OpenCF Summit is all about (hey, you already know what cf.Objective() is all about, right?). Seven Languages: Io, Day 2 - Unlike Ben Nadel, I don't intend to post my homework from each day of this exploration but I said in a comment on Ben's blog that I'd post some of my solutions. Today, I'm posting part of my Io Day 2 homework. fibhelper := method( a, b, c,
It's weird. This was one of the standard examples everyone always wrote in every new language they learned, back in the day. Yet faced with it yesterday, I drew a complete blank. I wanted to create an infinite lazy sequence of fibonacci numbers and then just walk to the Nth number. It actually took me a while to come up with the code above - and that was after several attempts at a straightforward loop. The recursive version is just the natural way I think, I guess!
OK, now I have the recursive version, here's my iterative version: fibr := method( n,
Meanwhile, here's Ben Nadel's version of the Io Day 2 homework. | |
| FeedSee | |