By Janice J. Heiss
Published January 2012
This series of interviews spotlights Java Champions, individuals who have received special recognition from Java developers across industry, academia, Java User Groups, and the larger community. (This is part one of a two-part interview with Dick Wall. In part two, we'll explore his current fascination with Scala.)
Dick Wall, a Java programmer for more than 15 years, began writing Java applets using Symantec Cafe on a Mac. He has worked with large simulations of energy distribution systems and with geographical information systems, and was an Android advocate at Google. Currently, he does research into human genetics using Scala at Locus Development. He also partners with Bill Venners at Escalate Software, where he provides Scala training and consulting, and currently he works almost exclusively with Scala. In addition, he is the creator and co-host of the popular Java Posse podcast, and he leads the Bay Area Scala Enthusiasts (BASE) in Silicon Valley.
Oracle Technology Network: You are a developer at Locus Development, which your bio describes as "a very small company doing some very hairy science and mathematics related to human genetics (in Scala)." You previously worked for Navigenics, a company that provides "genetic insights" for people to "motivate behavior change." Prior to that you were at Google. What motivates your professional changes?
Wall: In truth, the work we are doing at Locus Development is an extension (in terms of research and scientific completeness) of the work we started at Navigenics, which offers personalized genetic testing and analysis services. It’s safe to say that we are all believers in the idea of genetic analysis as an important component for people to understand and improve their health and quality of life. We’re building the capabilities that can empower genetic analysis, but for me it's more than that. The work is fascinating and very challenging in terms of the mathematics and the size and complexity of the data. In short, it's a seriously fun job.
I’ve always sought out jobs that are challenging. For me, that involves more than writing another large e-commerce site or another Web framework. Those things are fine, but they don’t interest me. I thought Google would be more interesting and challenging for me than it turned out to be, and I deduced that the only way to get the level of excitement and variety I want is through a small company.
Oracle Technology Network: Tell us what you hope to accomplish by using Scala for genetic analysis?
Wall: The general answer, from the Locus Development viewpoint, is a service to improve the breadth and quality of genetic information that a clinician can use in practice. I can't be more specific than that at present (or at least I’m not supposed to be). We hope eventually to produce something really good out of this work, and we have some of the best people in the world working on it.
Scala enables us to move fast. It provides a strong platform for mathematical analysis and handling large data sets and taking advantage of parallel processing. It also provides a lot of safety and security in the implementation. For example, programs written using a language that runs on a managed environment and for which you have made as much of the data immutable as possible, are less likely to succumb to an injection attack. Scala appears to be a good choice for the problem domain, and we have noticed several other well-known genetics-focused organizations choosing Scala as well.
Oracle Technology Network: Can you tell us how you use Scala in your work with genetics?
Wall: Scala is used for all aspects of development at Locus. It’s not the only language used, but it’s the primary one. We use it for relatively normal activities such as storing data models in SQL databases through analysis pipelines for designing primers (synthetic DNA), and we plan to eventually use it for massive computational runs on large data sets. We are not quite there yet, but Scala is ready when we are.
Oracle Technology Network: What have you learned about your own genetic vulnerabilities?
Wall: I went through the Navigenics service when I worked there and discovered that I really need to take better care of my heart, and I have an increased risk of macular degeneration (a condition that affects your retina). As a result, I exercise a lot more now, eat better, and take fitness more seriously. At the same time, I started eating more leafy greens (which should help with the macular degeneration), and I get my retinas checked during each eye exam for signs of trouble. I am happy to say there is no sign of it yet, and I intend to keep it that way. I can't wait to see the Locus Development service work its way through the healthcare system so I can learn more. I like data--it helps me make good decisions.
Oracle Technology Network: Some claim that the Human Genome Project has been a disappointment because since the genome was mapped, there has been little progress in applying this new knowledge to treat disease. How do you view the future of applied genetics to health care?
Wall: I would strongly disagree with that claim, but it's going to take some time. From a programming perspective, consider this:
The human genome is about three billion base pairs long, which is like having a 400 megabyte binary file with no disassembler and no manual. The best tools we have at the moment resemble grep, and the grep expressions take a few hours to produce a result. It's going to take a while, but we won't get there any faster by ignoring the effort or calling it a failure. At this point, we think we know some of the more important parts of the binary, and we can occasionally find locations that really make a difference, but over time, the whole field is going to improve. It's just slow going.
The first thing most doctors want to know is your family history. That's because health risks are often inherited from your parents through your DNA. Family history is a broad brush with which to paint a genetic picture of your risks. We are trying to narrow those brush strokes and make them more accurate and precise.
Oracle Technology Network: What should Java developers understand about genetics and information technology?
Wall: Probably nothing. But, for those interested in genetics, I can recommend the book Genome: The Autobiography of a Species in 23 Chapters by Matt Ridley, which is very informative and entertaining. Apart from that, they can google for genetic-related subjects and check Wikipedia.
Oracle Technology Network: What can you tell us about alternative languages for the JVM?
|“One of the problems facing projects that mix many different languages is how to get them all to build together. When you mix languages, you can end up with a chicken and egg problem of what to compile first.”
Wall: I have used many over the years. I have flirted with Groovy, Jython, and JRuby, in particular, and I am focused on Scala now. The JVM has really proven its flexibility in being able to host so many diverse languages. One of the problems facing projects that mix many different languages is how to get them all to build together. When you mix languages, you can end up with a chicken and egg problem of what to compile first.
For example, if you mix Groovy, Scala, and Java on the same project, and you have each calling the other languages, resolving what gets compiled first can be really hard, and without that ordering you can't ensure that everything will actually compile properly. Scala understands enough Java to be able to generate the symbol table for the Java code, so if you compile everything with Scala first and then the Java files afterward, that works, but it gets harder when you mix in more languages.
Andres Almiray has had success getting Groovy, Scala, and Java working on the same project with Griffon, but for a more complete solution, we might need some kind of organized, two-phase pluggable compiler that everyone agrees on. The first phase would give all languages a chance to generate their symbol tables and do as much work as they can. Then, the second phase would complete the compilation for all languages based on those generated symbols. It's a lot of work, and many people would have to agree on the standard, so I’m not holding my breath.
Oracle Technology Network: Give us your assessment of the new generation of languages for the JVM such as Ruby, Python, Clojure, and Myrah.
Wall: Ruby and Python are established languages in their own right, and they have done very well independently of the JVM and run really well on the JVM. Clojure is interesting and has lots going for it, particularly a very strong influence on immutability and the software transactional memory model. Being a bit of a reformed Lisp developer though, I worry that Clojure will end up being the basis of a million dialects eventually just like Lisp was. Myrah, a statically typed Ruby-influenced language that runs on the JVM, is also interesting, and I think it validates the value of static typing even for a traditionally dynamically typed language. There are so many others out there I won't list them all.
|“You will never waste time learning another language. It's always going to expand your range and skills as a developer, even if you don't end up using it day to day.”
You will never waste time learning another language. It's always going to expand your range and skills as a developer, even if you don't end up using it day to day. At present, I am digging in to Haskell since I think that’s the best way to round out my functional programming skills. In the end, though, Scala was the first language to hit all of my pain points at once, and I suspect it will be the language I continue to use for the foreseeable future.
Inside the Java Posse
Oracle Technology Network: Is there any inside scoop you’d care to share about the Java Posse?
Wall: We drink quite a lot when we record our show, but most of our listeners already know that. It's still fun to do after all these years, and despite what it might sound like at times, we are all firm best friends, so good, in fact, that the arguments we have are usually because of just that. We know each other so well now that there is little we can't say to each other. I wish I had more time to do it and we could get back to weekly episodes, but there is just too much going on in life right now.
Oracle Technology Network: What is the strangest thing to happen on the Java Posse?
Wall: Strangest in my memory is probably our first live recording -- at JavaOne back in 2006, I think it was. We didn't expect a lot of interest since podcasting was fairly new, and we weren't very well known. In fact, we were surprised that we got the approval to do the recording. I remember being super-nervous that no one would show up or that, if they did, they would hate us. It turned out to be standing room only, and the crowd was great and very funny. They spontaneously broke out into song at the beginning, and that set the tone for the rest of the show. The walk through the streets of San Francisco afterward, with a huge group of people, to go to Katie O'Brien’s was pretty strange, too. Of course, we were wearing the big floppy cowboy hats, and I remember Joe dancing with a slightly confused but enthusiastic guy on the street, still wearing his floppy hat. That was all pretty strange. Stranger still is the cult of personality that surrounds the posse. We are just developers who record chats we have every couple of weeks. What it has become, at least in the Java space, is as surprising to us as it is to anyone else.
Oracle Technology Network: Tell us about your Flubber program.
Wall: Flubber is a simple tool I created to mark mistakes when we record the podcast. It’s been through various incarnations. The one I use most now is the Android application I created. It saves a lot of time in post production, and it is still used for every Java Posse episode. It's also freely available to anyone who wants it, but unless you are recording a podcast and using Audacity to edit it, it's probably not that useful. For me, it's a life saver.
Oracle Technology Network: What has surprised you the most about the evolution of the Java platform?
Wall: The enduring nature of the JVM has to be the most surprising (and pleasing) thing. The ecosystem that has been built up around the Java language and the JVM is also very impressive. Other development ecosystems would love to have the diversity, energy, and size of the Java and JVM ecosystem.
Oracle Technology Network: What are some things you would like to accomplish using Java?
Wall: All technology exists (or should exist) to improve quality of life. People seem to forget that sometimes. “People over technology” should be the message, so anything that improves quality of life is a good thing in my opinion.
Oracle Technology Network: If you had total control over the future of the Java language, what would you want? What would you want from the JVM?
Wall: I outlined many of the big items in the article I wrote for Java Magazine(http://www.oracle.com/technetwork/java/javamagazine/index.html) about Scala and the JVM. Tail calls and type reification would be very nice, but we are doing okay without them, and I would prefer that they were added correctly rather than quickly. As for Java, I do think it's time that function literals and closures were added, but after that, maybe it's time to take a look at what other JVM languages have to offer rather than trying to put more into Java.
Oracle Technology Network: Do you listen to music when you code?
Wall: Yes, I listen to all sorts of things. I have a pretty diverse collection (classical through to thrash, with lots in between), and often one day I will not want to hear something I loved the day before. Music gives me a distraction for sneaking up on more difficult problems, the ones that you get stuck on if you just try to hit them head on. Sometimes I work in silence, but much of the time I have some music going.
Oracle Technology Network: What do you do when you are stumped?
Wall: I like to ride my road or mountain bike when I am stuck on a problem. While in Crested Butte last year, I solved a major problem I had been stuck on at Locus while I was half way up a mountain wearing snowshoes. My wife Jackie noticed the enlightened expression on my face and said, "Did you just solve a problem?” So she knows me pretty well at this point. A change of scenery is highly underrated when it comes to problem solving.
Oracle Technology Network: Where in the process of programming do you have the most fun?
Wall: I think it is in the diversity of tasks. I get bored quite easily, so I like a lot of variety. It's always nice to see something you created from the ground up actually working, and it's even better when the people you wrote it for like it. Sometimes I get enjoyment from tracking down a really tricky bug, and other times I love having a really tough problem to design around. I think I like most aspects of software development when it comes down to it.
Oracle Technology Network: What technical insights into the Java programming language have been most important to you?
|“Pretty much everything I thought I knew about optimizing for the JVM was wrong.”
Wall: Talking with Brian Goetz is always fascinating. Pretty much everything I thought I knew about optimizing for the JVM was wrong. For example, when I started working in Scala, I was worried that lots of extra immutable objects, which are created when you use immutable data often, would result in a lot more work for the garbage collector. After talking with Brian about it, I realized that, in fact, the opposite is often or usually true. Short-lived, immutable objects usually exist in a special part of the JVM’s memory referred to as Eden. Releasing the memory back to the pool from there is almost without cost. It is only longer-lived objects that get promoted to the JVM main heap that are expensive to garbage collect. So lots of small, short-lived objects can actually help the garbage collector out. There are other ways immutability can help or hurt performance, but ultimately, I decided to code for style and correctness first and worry about performance if and when it becomes an issue. Brian usually advocates this approach, too, and in fact, I almost never find myself worrying about performance any more.
Oracle Technology Network: The Java class that you couldn't live without is...?
Wall: I think, from the standard Java libraries, ConcurrentHashMap is pretty high on the list, although I think Guava MapMaker is even better. In Scala, the collections classes are my favorites, although I would have a tough time picking just one.
Oracle Technology Network: What recent changes to the platform have made your life more complicated?
Wall: Generics were introduced in Java 2 Platform, Standard Edition 5.0 (J2SE 5.0), and they were generally a good thing, but I do think that wildcards made things harder than they should have been. That was some time ago, though. There haven't been that many changes to the platform more recently, and none that I can think of have made things worse. The loss of some kind of closure solution in Java Platform, Standard Edition 7 (Java SE 7) was a bit of a disappointment, but by that time I was working pretty much full time in Scala, so it didn't hit me that hard.
Oracle Technology Network: What's the most fun you've had working with Java technology?
Wall: If you include Scala, then definitely the work I do for Locus Development qualifies. Working on human genetics is tough to beat, because it's such an interesting field. For Java, I probably had the most fun with the work I did at NewEnergy Associates, where we did some pretty large simulations or calculation tasks that were often carried out in Java.
Oracle Technology Network: Can you describe the process of writing code?
Wall: I'm not sure this is the answer you’re looking for, but I try to learn something new every time I write a piece of code. So the process always involves learning for me. I also like to visualize a solution in my head before committing to it. One trick I use a lot in Scala is to use the REPL (the interactive shell) to try out ideas as I think of them. I use that to put together a rough prototype for harder problems, and then I commit that into code.
I also use “test first” sometimes. I am not religious about it, but if I am working with a domain expert who has have given me some good, specific goals that I can write tests for, I do so and let that drive the rest of the development. I always enjoy doing that, but I find that approach doesn't fit all tasks. I don't believe in silver bullets for software development; I believe you should rely on a broad arsenal of problem-solving approaches instead.
Oracle Technology Network: What would you like to see in Java Platform, Enterprise Edition 7 (Java EE 7)?
Wall: The cheeky answer is that I would love to see what someone could do with a Scala Enterprise Edition. In some ways, libraries such as Akka are already closing in on similar functionality, and I think we might see more complete solutions for enterprise developers using Scala in the next year or two.