The Conj
Below I’ve interspersed the wide variety of ideas I came home with from the first Clojure Conj with my own thoughts. Notes I took from the conference you’ll find in black text and you can be reasonably assured that someone said them and I wrote them down. I didn’t take notes on every talk, and of the ones I did, I’m sure I missed some details or got important ones wrong, so feel free to correct factual errors in the comments. Connections between talks and ideas I heard echoed throughout the conference I wrote in green. My own commentary and interpretations are in blue. There were a lot of links and books, projects and presentation materials that sounded interesting, I’ve collected them all as part of my clojureconj materials page, and referenced them throughout this writeup.
The article itself is fairly long, here’s the talks I cover:
Fertile Ground: The Roots of Clojure – Michael Fogus
Zippers – Luke VanderHart
(not= DSL macros)
Lisp, Functional Programming, and the State of Flow
Protocol XIII: Clojure Protocols Explained
Finger Trees: Custom Persistent Collections
Making Leiningen Work for You
New Features – Rich Hickey
Hallway Track
Lightning Talks
Keynote – Rich Hickey
Simplicity Ain’t Easy – Stu Halloway
Fertile Ground: The Roots of Clojure – Michael Fogus (@fogus)
Fogus talked about the ideas and languages that influenced Clojure. They were wide-ranging and not all directly drawn from the field of computer science. He gave a particular shout out to Ruby, Javascript, and Python as languages that prepared the world for Clojure. His talk was full of good stuff, so he didn’t dwell on this point, but it really struck a chord with me that Python in particular legitimized dynamically-typed programming. That’s a very broad statement and I’m not qualified to defend it, so I won’t, but I do believe that had things gone a little differently I might never have written a single line of interpreted code.
Questions I didn’t think to ask after Fogus’ talk:
“What will Clojure’s influence be? Will it be new ideas, or just a combination of the best old ideas? What will a language influenced by Clojure look like?”
Bonus followup: “Did Java not introduce enough new ideas and is this why there’s such a vacuum left in it’s wake?”
See also [1] and [2] as extra reading, both referenced in the talk.
Zippers – Luke VanderHart (@levanderhart)
Based on a paper [3] by Huet, Luke took us through Clojure’s zipper data structure. To create a zipper, you must supply 3 methods. For certain data structures like nested lists and vectors, these methods are very easy to implement.
Navigation and editing become a one-liner:
(-> t down right down down right right (edit f) root)
Zippers incur low creation overhead, through a really neat strategy of storing it’s “implementation” in metadata. Since many structures are navigable as zippers, the library adds metadata to the existing data structure via with-meta. This approach works because clojure structures are immutable and you don’t need your own copy of the data that only you control. Additionally, by not storing the functions in an outright data structure, less data is copied from version to version.
There’s no reason that the 3 functions you provide the zipper have to operate on existing structures: the functions could generate data on the fly if you are navigating a regular, predictable structure.
(not= DSL macros) – Christophe Grand (@cgrand)
- Users don’t actually want a DSL.
Rich echoed the sentiment that users don’t know what they want in his day 2 rant by saying not to allow users to ask for features, instead have them tell you what their problem is. I would add that even then, most of the time they don’t know what the actual root problem is, so you can’t listen to them about that directly either, you have to figure it out for yourself. Users are pretty good at describing the symptoms their problems are causing, though. - Users want the full power of Clojure to manipulate “pieces” of a DSL
- Values are a sweet spot, design-wise
- full power of Clojure
- macro-enough that expression is terse
- Macros are icing, not first steps
- Map domain-specific semantics onto datatypes/values
- Hide unmappable abstractions behind deftypes (in his presentation, Grand used a defrecord, but Rich during hallway discussions and Stu via twitter chimed in that types are abstractions, records are solely field/information holders)
- You have an abstraction/semantics creator function + an “eval” function to do something with that abstraction
“Each of these slides is gold. If one doesn’t teach you something, you haven’t looked hard enough yet.” – Chris Houser
A few things he mentioned that piqued my interest are under my links section: [5] [6] and [7]
This was one of my favorite presentations, go download his slides [19] right now and forever more build your libraries like he does, Clojure will be better for it.
Lisp, Functional Programming, and the State of Flow – Tom Faulhaber (@tomfaulhaber)
Tom talked about what flow is as Mikhal Csikszentmihalyi describes it, as well as masterfully pronouncing the author’s name right on multiple occasions without missing a beat.
His first example was how natural and useful fill-queue [8] was.
His examples illustrated how Clojure lets you write at whatever level of granularity you’d like, which he had mentioned earlier in the talk was important for flow. I made the connection as he was talking, but I don’t think Tom ever really made that connection explicit nor did he instill any larger realization/connection gained from that insight. The gem of the presentation was an excellently specified problem of providing near realtime updates of commits to a source control repository. His solution was elegant and he took us through it, but the problem was so perfectly described and the proposed solution was so natural, that I feel like I could have written the code myself in a short time.
Having a clear written problem statement that is continuously updated would be incredibly useful to a team.
Later, during the lightning talks, one on Aleph [9] mentioned (and I agree) that Aleph might have been an even better match for solving this problem than the solution Tom showed us.
Protocol XIII: Clojure Protocols Explained – Sean Devlin (@fulldisclojure)
Very cool talk here on ad-hoc type inference via protocols. He implemented a protocol named Dateable and chose a single method to unify/describe all Dateables (don’t underestimate the complexity of this step, it’s important to choose a good method). He then wrote implementations for lots of concrete types. Combined with custom “cast” functions, he showed how you can then "use the right type for the job"
This is very powerful way to work with a bunch of disparate types in a single generic way, without losing the ability to use specific features of any type regardless of the “underlying” type. This really blew my mind. Given a long, you could treat it as a Joda time.
Learned during the Q&A Session:
reify creates an unnamed type (an anonymous type), use extend if you already have a type that you’d like to imbue with the protocol’s abilities.
Someone asked if you can extend one protocol to another. You can’t, but when this question was asked, Rich Hickey jumped in and noted that if you wanted protocol P to extend A, because you can’t write:
(extend P A (method [this] ...))
you can extend Object to a protocol A, which will be a default fallthrough case. Given the object, you can test if it satisfies protocol P and if it does, you can then extend the object on the fly to implement protocol A in terms of whatever facilities P offers. This is a technique that I’m sure we’ll see more of when Clojure protocols outnumber Java classes in variety and usefulness.
Finger Trees: Custom Persistent Collections – Chris Houser (@chrishouser)
Affectionately known and referred to throughout the conference as the single-syllable “Chouser”, Chris gave a detailed and remarkably easy to follow talk on what appears to be an incredibly dense and detailed data structure. Go download the slides, they do more justice to his talk than I could. The idea of finger trees came from a paper describing them in Haskell. Chouser eased us into thinking about them by introducing a specific case that makes dequeues very easy. In the general case, a more complicated function called a meter is used under the covers to make common list & tree operations inexpensive.
- Every node is decorated with a count of the elements contained beneath it
- Each level of the tree indirects once more than the previous
- Leaf nodes hold no more than 4 elements
- The fixed number of nodes (< 4) means that
nthcan be written as a O(log n) operation (vs. O(n) ) - Splitting trees is easy, and the ability to quickly add a node to the "tail" makes arbitrary concat/modification cheap (reminiscent of zippers). Since split and concat are both log n operations modification in the middle of a finger tree is O(2 * log n + C), which has O(log n) growth characteristics
Making Leiningen Work for You – Phil Hagelberg (@technomancy)
The /checkout folder is functionality (already in? currently being added to?) lein to add features to one project that your current project will use without doing the maven dance.
Phil mentioned some community-building concepts that lein has benefitted from:
- LHF – identify low-hanging fruit
- document internals and maintain a hacking guide (much easier if you start on day 1)
- accept patches promptly
- trust people, commit rights should be easy to get
New Features – Rich Hickey (@richhickey)
All new features are in master on github, most were committed within the past few days of his talk, but all had been brewing under that delightful mop of hair for much longer.
BigInteger contagion. Rich talked about Clojure’s primitive support, the language’s new rules for handling primitive types as parameters to functions and return values from functions. To implement primitives as functions to parameters, Rich added a bunch of sub interfaces whose names reflect the types of values they take and return. IFn$ODL takes an Object and a double and returns a Long. He used a one-off Clojure function to generate the Java code of each combination out to 4 parameters of each type. He wants feedback. He encouraged Clojurians to write numeric code, and tell him if the the IFn$LOL methods are enough? Does this bring performance acceptably on par with Java? Rich also said that all Clojure programmers care about this. Because even if you don’t write numeric code, this sort of thing creates buzz at the top of the budget food chain that Clojure is a contender and that it doesn’t have to apologize for it’s performance, which tends to trickle down and green-light web projects or tool one-offs. This work was a strategic as well as a tactical improvement.
Rich is trying to get rid of annotating things as static as a way to make Clojure fast. He’s not sure if he’s there yet, so static is still around, but it sounds like it’s never been an optimization he liked adding and he’s actively trying to kill it (as when it’s dead that will mean Clojure is fast without tricks).
Clojure hasn’t gotten every detail and design decision right…yet. Rich talked about binding and threads. Currently they don’t work seamlessly together, which is a shame because less and less work is being done is a single thread. Lots of features in clojure spawn new threads and to not have those features work well with binding is a sore-spot. Ultimately, Rich noted that on the horizon are resource scopes: collections of threads, processors, files and more that are allocated as a unit of work and then released when the work is complete, decoupled from what thread things are running in or other orthogonal concerns. For now, Rich has made def work differently. (Again, on master. This work is yet unreleased). Dynamic binding has a cost (Rich started a little Socratic back and forth with the audience, prodding us to tell him how dynamic binding was implemented) and most people are paying for that cost without using it. Rich decided to change the way def works to remove that cost. def will no longer make dynamic vars by default, thereby removing the cost for the vast majority of def‘s usage. Dynamic variables will need to be declared as such, thereby maintaining the usefulness in scenarios which it is desired. The use case for dynamic variables is more and more shifting towards simplifying value-access in multi-threaded code.
In this session I learned the term “earmuffs” as descriptions for the stars around variables like *out*. (assoc brain that-info)
Also Rich mentioned, and I quickly wrote down, that to mock functions and globals during testing, you should use alter-var-root, execute the test and then alter the root back, DO NOT use binding.
Hallway Track: 200+ simultaneous presenters
I participated in a lot of fun discussions outside of sessions. Some even had something to do with Clojure or software in general.
If you want to be different, use cake/maven bitbucket and vi. The build tools, source control repos and editors Clojurians use are wide in their variance, but the overwhelmingly chosen toolset seemed to be lein, github and emacs.
I talked to some guys from Forward, a company that describes itself as “post-agile”. Another person in the conversation mentioned that he was post-agile, as well as post-OO. It struck me that both approaches were honest movements created to address real needs and presented solutions that solved real problems. However, there was a lot of disillusionment when those movements applied unilaterally failed to lead all programmers to the promised land. The ideas taken as far as they could failed their most devout practitioners. Some refused to acknowledge this, some saw it and rejected all aspects of the movements on those grounds alone.
So it is that today we are “post” agile and OO. The special irony to my ear is that agile, as I was taught it, was a methodology to eschew all methodologies. XP and TDD and Kanban and Scrum all proscribed certain ways of getting things done, but the underlying core of agile seemed to me to already be post-agile. I don’t think that message has come across loud and clear over specific failures and general disillusionment with the practice over time.
Day 2
Infer
- Born of a mix of research (one developer is a doctorate?/post-doc? at Stanford) and practical (the other founded a startup in machine learning and big data) That’s an awesome combo to be on either end if you can swing it
- For linear and quadratic programming, the hard stuff that you can’t make run in O(log n)
- "a lot of people have numerical optimization problems and don't even know it."
- If you do know it, the Infer guys are interested have made something for you and are interested to hear from you about the problem you are solving
- Check out [11] and [18]
Clojure Debug Toolkit
- Effort to use more of the JDI
- Debugging was mentioned in Bedra’s talk as a real Clojure sore-spot
Keynote – Rich Hickey (@richhickey)
This is not a methodology. It’s a rant.
Solve problems with minimal typing.
Solve it the first time you sit down to solve it (or at least you should have a chance to).
Most of the biggest problems in software are of misconception. This statement says more to me now that I’ve thought it over for a while than when I first heard it. It says that problems in the field of software are not limited to those that a computer can solve, which implies that you solve a lot of problems in software without a computer. This talk focused on a sub-category of those problems. Also, this means that software itself really only solves a single class of problems: computation, which I think is a helpful way of thinking about what you’re really writing
The step before go do it is an important step.
You will get good at what you practice. Almost without trying. You can practice thinking about things, but maybe not in the way would you practice a sport or a hobby.
Say the problem aloud. Write it down if you have to. I tend to think that thinking it is just silently saying it, but it’s not. Writing it down might not even be. There is a different part of your brain that switches on when you know someone else will hear your words
Write down, examine, and think about the characteristics of the problem. What do you know? Describe. Stu Halloway’s closing talk noted that approval and disapproval are rampant in modern language, and previously precise words that used to describe things tend to lose meaning over time as they are used more and more genrally to approve or disapprove of things.
Lots of input. You must feed your brain. Bake your ideas. Your forebrain is tactics, but your unconscious brain is strategy. The forebrain is good at reasoning after connections have been made by the unconscious brain.
Caveman forebrain: “Huh, look at all those Bison down by the water”
Caveman unconscious brain after a good night’s sleep: “We should hunt near the watering hole”
Caveman forebrain: “Yeah, duh. That’s what I said”
Don’t think that your forebrain will be all that good at creating new ideas. the fact that it can decide if a good idea holds together or not leads many to think that their forebrain can create good ideas, they’re two separate abilities
"When the facts change, I change my mind. What do you do sir?" – John Maynard Keynes
Q&A
Someone asked a question about bad ideas and getting too attached to them. Rich said to kill them, throw them away. Stop wasting time on them as soon as you know they’re bad. "they're not your children"
Rich singled out a book by Polya [12] as excellent material for practicing the skill of solving problems.
Simplicity Ain’t Easy – Stu Halloway (@stuarthalloway)
In between pulls on a frosty bottle of Newcastle, Stuart Halloway dropped some knowledge on the meaning of simple.
As a concept, it’s not subjective this is a very strong statement in today’s culture of anyone-can-say-and-argue-for-anything-on-the-internet. It is very tempting to look at the sheer number of ideas, realize how little you know — or could ever know — about any of them, and give up and think that there might truth in all of them. Stu is not just saying that simple is not a subjective topic, but that there exist topics that are not subjective. There is right and wrong in the world. Black and white. It doesn’t help that the colors change as we learn more, but don’t give up and don’t bow to sheer pressure or overwhelming opinion. In other words, when someone is wrong on the internet, they might actually be wrong).
"people are far more anxious to express their approval or disapproval of things than to describe them" – C.S. Lewis [15]
Simple has been watered down through this long human drive to approve or disapprove of things. This strikes me as a cognitive reaction to ambiguity. As a being of finite mental capacity, you have to accept that there exist many things you will know nothing about — could never live long enough to know everything about — and you have to short-circuit to make a decision to act on or trust information to be true somehow. Approval is an excellent way to do so, it’s not very rigorous and it can be transferred through networks of trust)
I had to duck out to catch my flight just as Stu started to apply these insights to Clojure. But I rather like that I didn’t hear the points. At least until the video is put online, I’m going to think of it as an exercise left to me to make my own connections.
Two Guys Arguing 
4 comments