JAX London and Big Data Con are just a few weeks away and naturally, we're very excited about what's in store for the events. It seemed like the perfect time to catch up with those keynoting at the event itself in what we're calling the "Road to JAX" series, which will cover some important topics for the worlds of Java and Big Data. Next up is the delightful duo of the Diabolical Developer and LJC co-leader, Martijn Verburg and Java Champion and performance tuning guru Kirk Pepperdine. The two will be speaking just before JAX London Community Night with their keynote 'Java and the machine'. We caught up for 30 minutes to talk all things Java (cloud, modularity and mobile mostly), JClarity (the Java performance company they've just launched with Ben Evans) and some important community stuff. Enjoy! ------ JAX: Obviously, you're both keynoting at JAX London with 'Java and the Machine'. Without giving too much away what is it about? MV: Basically, the physical environment that Java developers, and that Java itself is running on, has actually now changed in the last couple of years and is continuing to change. We see two major changes. One is the fact that CPUs have gone multi-core, which means the program runs very differently under that piece of hardware. Also, the fact that a lot of people are moving toward virtualised operating systems, which means a lot of people are no longer running on what we call 'bare metal'. This is a new paradigm that Java developers have to get used to and take advantage of, as well as Java the language and platform itself. We're going through this quite interesting period of change. KP: You've put a positive spin on it, I'll put a negative one on it. Hardware is changing and a lot of that change is for the better. It is making lives more difficult for developers because they have to think about things they didn't have to think about before. However the platform running on, in terms of the virtualisation, is actually - well you know, the better they make the hardware, the more things we can think of to consume it. A lot of performance challenges when you move to a virtualised environment. In other words, it's not a straight bare metal to virtualised environment switch and everything's going to work the same. It doesn't work that way. I like to tell people that there's plenty of good reasons to move to virtualised environments - performance isn't one of them. MV: Absolutely not. KP: There's many cases of moving to virtualised environments is going to cut people off from some of the advances made in hardware, and make it more difficult to get at things and make their deployments prone to different types of failure. So it's not completely a panacea - although there are some obviously advantages to virtualised environments. There's a lot of people to be solved there. JAX: Do you think developers don't embrace multicore now, will they get left behind? MV:They will be left behind, absolutely. Dead in the water. It's adapt and survive and don't adapt and die. It's Darwinism at its best. JAX: Simple as that? KP: It's worse than that - they've already been left behind. We've been telling them from 2005 to start adapting. In seven years, if they haven't started adapting, you wonder what they're going to do with the new changes in hardware and the new resources available. How are they going to even consider taking advantage of those resources? I'm not saying it's an easy slog to adapt to multicore environments. There's a lot of interesting intricate problems that require a lot of thought in order to work your way through them. Our simple pessimistic lock-on type models we've been using in the past - they simply won't scale. We actually have to be more creative. Fortunately in Java, we have an advantage in that we have these very clever library writers helping us. A lot of developers can accidentally move to multicore. JAX: Do you think a change of mindset is needed - is it as simple as that or is there more to it? MV: It's not just a change of mindset but a willingness to knuckle down and practice. There's the old adage that you won't become a world expert unless you do something for 10,000 hours and we're not saying they have to go to those extremes. KP: Sure they do! MV: (laughs) Kirk would argue otherwise. If a number of developers have the discipline to sit down and learn even the existing libraries, which have been there since the mid 2000s, [then] it's very much a case of knuckle down and learn the stuff or be left behind. JAX: Do you feel like Oracle are doing enough to embrace this issue? KP: Absolutely. MV: I have to agree. They're putting in an awful lot of effort. If you look at the work being done on Java 8. Lambdas isn't simply about reducing a nice bit of boilerplate source code for developers and not even necessarily all that much about adding the functional programming aspect to Java. What it's real power is is enabling parallelism inside a lot of the default collections, libraries, other APIs inside Java itself. All of sudden for completely free, developers are going to have collections which work really nicely on multicore machines. That takes a massive amount of engineering effort on behalf of Oracle. KP: It even goes historically before that. When people were predicting that we're going to have trouble with what to do all these transistors, Oracle looked at this and said "Crap, our database speed is coupled to clock rates." If there's no more free ride in getting high transactions based on clock speed improving, what are we going to do, right? They started a long time ago, even pre-2000, looking at what investments needed to be made in order to continue their dominance in the database space. I think that they've been thinking about this for a long time. Sun had been thinking about this for a long time. If you look at the post-Sun era moving into Oracle, they're continuing to think about it. It's been going on for a long time and will continue to go on. If you get a brand new piece of hardware, you come with 800, albeit limited cores on your machines that aren't utilised. Oracle's already started a project saying "How can Java take advantages of those 800 cores sitting there basically drawing triangles." JAX: Do you think this is the biggest challenge Java faces - is this the one? MV: In terms of Java on the server, I think it is one of the biggest ones. In terms of keeping Java the language and platform at the forefront, I guess, [at the height of] consumer awareness, Java needs to have a stronger story in mobile. At the moment, you've got Android, kind of Java-based but we all know the conflicts over that. That's probably the only piece puzzle missing. Modularity would have helped there, so Jigsaw. It's a real shame it got delayed, I do think it was the right call to delay because it simply wasn't baked in well enough. It's certainly one of the biggest challenges Java faces but I actually think this side of it is going reasonably compared to the others. KP: I think we're far ahead on that front and there's no doubt that something very disruptive could come along change the picture very quickly. I haven't seen what that is yet and certainly, that's not where the problems are. The biggest problems are the apparent lack of presence on mobile devices. JAX: Is that the key hindrance for you then? KP: I wouldn't say it's a hindrance. When I look at the future of computing, I don't see laptops or desktops there, per say, but I see server rooms and people running around with mobile devices, or even something more mobile than a mobile device. In those cases, you have to say that when it comes to the big iron as Marty has pointed out, Java is a clear obvious choice there. But when you move into the mobile space, I would even be as bold to say, I'm not sure the VM technology as it is implemented today, as it works against the hardware today, is actually a good solution for mobile devices. I think that's going to be a struggle - how do we change the technology so that it does fit in that particular space? I think Jigsaw would have helped, but it's a very small piece of the puzzle. You have to look at how the VM works and that's what I think Google did when they looked at Dalvik and they said how can we make this better for a mobile platform. They come up with their solution - is it the best one? I don't know, it's certainly better than the alternatives at the moment. JAX: Let's move on to JClarity. How did that come about and the goals behind setting that up? MV: It's a kind of a convoluted story. There's three of us - Kirk, myself and Ben Evans. We've known each other quite some time on the conference circuit, mailing lists, talking about the JVM and performance and also fun tech stuff. A couple of years ago, Ben and I were working on a book together and we decided that was going quite well and we wanted to start a company together. We kind of bandied around some ideas but nothing was sticking, all that much that we felt would be an industry changing or industry significant idea. At that time, Kirk had kindly invited us to teach his performance tuning course. After teaching that a couple of times, a lightbulb went off in our heads, and we thought Kirk has actually done this for so long and got this brilliant methodology which despite being slightly experimental, kind of worked. KP: Kind of works? MV: You can turn it into software. We approached Kirk straight away and said "Hey, want to turn this into software?" and he turned around and said "I've been waiting to do this for 10 years, why didn't you guys ask me earlier." That about sum it up Kirk? KP: That about sums it up JAX: What is the raison d'être of JClarity and the company moving forward? KP: Really in my mind, it's about disrupting the performance space. It's not a black art, or magic. It's not unpredictable, there's a lot of predictability here. There's a lot of things we can do to actually further, for the lack of a better word, the science of performance tuning so that we can actually tool a lot of this stuff, so we can basically "professionalise" the space. MV: Turn it into a empirical science - we believe that it is and we've proven that it is. Kirk's passed 15 years of teaching and putting this stuff into practice, and Ben and myself and have been doing for quite some time. There's actually a small sort of loose community of us who have always had this feeling and it never really banded together. Now it's starting to happen. JAX: Do you think a lot of the work is to dispel myths for Java developers - are some developers wary of trudging into those waters without making too many mistakes? MV: Absolutely. I was just teaching this week in London and I had six very bright Devops/developers. All tech leads and senior developers. Lightbulbs were going off all the time. They'd go "But I read this blogpost that said to do this or my admin manual told me to do that, or my old mentor told me to do this." They never actually sat down and just measured things like any normal scientific process. Once they started doing that, with the experience and intelligence they naturally had, they were up and flying in a couple of days. And they were all really excited to go back and apply it in their day jobs. KP: We don't really teach anybody anything new - it's about refocusing how they think about problems. That's the most useful thing they come out of the course with. Once you get the right measurement, it'll give visibility into what's going on and if you don't have that, you don't have the right measurement. You just need to learn how to get it. JAX: I'm guessing some developers don't have that vision already available to them? KP: I've had conversations with a number of departments at universities that were asking "How do we teach this because no one teaches it?" Performance tuning is something that isn't taught. It's expected because we're in this industry where everything blazing fast. We're supposed to know it just naturally and what I find is that it's not a skill developers come out of school naturally, so when they actually have to go do it on the job, they'll do it. Some people will be naturally good at it and other people will come along and be less good at a particular aspect. They'll run into roadblocks. People come up with their own way of doing things. What we're trying to do is get past these naive approaches, and say here's some best practices, filtering them over time to figure out which ones don't work. We toss out things that don't work and even our process has to adapt to our changes in hardware and technology stack. Because when things do change, we're going to find there's modifications that we might have to make to the process in order to keep up. JAX: There seems to be a lot surrounding the cloud on your website, right? MV: The reason we're aiming for a cloud - actually it's a multitude of reasons. One of the big reasons we've seen the trend for a while now that Java and the JVM is going to dominate on the cloud going forward. It's really the only platform that can handle the scalability requirements. However, there's still this element of Java performance tuning being a black box and what we really hope to do is give illumination, clarity and light to people who have to administer hundreds, thousands of VMs. That'll hopefully make the entire cloud story a bit more stable and better for everyone really. We want to become the heart and soul of these cloud platforms. KP: If I could be so brazen to say on the scalability issue - there isn't a product out there that will scale into the cloud as we expect cloud deployments to scale. That's the first problem: the second is that all the tools require an extrordinary amount of knowledge of all the different bits of the systems. If you combine those two things, it really means, you've got a lot of problems with the current toolset. We think that it can be done better. We certainly want to be scalable so we understand the problems that cause things not to be scalable. So we're trying to avoid them up front. People to have to be rocket scientists to have to work tooling. JAX: With low-impact? MV: Absolutely. We sit on the other side of the fence when we've been working at enterprises in our careers. We've had vendors come in with these performance tools and they'll make claims like "this will have a tiny footprint on your application" and every single time, the performance tool becomes the performance bottleneck. It's really frustrating and we've flipped it on its head, and said this is our first requirement. Not getting at the data, not looking at pretty graphs - our first requirement is to make that whatever we build genuinely has very little to no impact when it runs. KP: I like to say that we're a guest on our customer's hardware and we want to behave that way. Low footprint, low impact. A lot of tools claim to have it. When you actually look at what they're doing, you find that they can get moderate amounts of scalability in a lot of cases, and some can scale relatively pretty good. But when you get into larger deployments, they start falling over. JAX: Do you think enough's being done in core Java development to work towards the cloud? A big question, I know... MV: It's a big area - they're certainly thinking about, but I guess even the really smart engineers at Oracle and OpenJDK are going into the deep unknown a little bit. We're not convinced all of the right work has been done or right decision are being made. But then again, who does really know? No one's done this before. Google talks about their Go programming language, which in theory, is being designed for a lot of these problems, but even that hasn't been proven yet either. There needs to be some exploration, and we're hoping the OpenJDK and Oracle are going to be reasonably flexible and try out ideas and use the community as much as possible, to actually test ideas out really early and fail fast. Then go down a direction which will help Java and JVM scale in the cloud than it already does. JAX: With the London Java Community being on the JCP, are you happy with the progress being made with JCP.next, Martijn? MV: Very happy with the progress. We set out an internal goal as part of the LJC User Group, of changes we wanted to see, in the standards in general around Java. For us, we're about a good 5-6 months ahead of the curve where we expected to be, so we're really pleased. A lot has been driven by Oracle and the other big vendors, and that keeps surprising people. They want to open things up because I think they realise the value that the bigger they make the pie, the more everyone gets a bigger slice. It's in their best interests. JAX: Would you like to see more JUGs on the JCP? There's two at the moment in SouJava and LJC - are more needed? MV: I actually think the balance is quite good with two of us. We did all vote together to reduce the size of standards body. Too many cooks spoils the broth, a decision never gets made. The Executive Committee is being reduced to between 20-25 people. If two are User Groups, or at least representatives from the community, then I think that's a decent representations. JAX: Adopt a JSR, it's been a year since that initiative started. How's that going and how is it moving forward? MV: It's going really well. We've actually got a whole swathe of events at JavaOne, in and around Adopt A JSR. The community keynote will touch upon it for 10 minutes, and there's workshops etc. At the moment, it's probably at the stage where we've got a few large Java User Groups around the world, which have all banded together and are working on these things. The next stage is we want this to be applied. At the moment, the Chennai JUG in India, SouJava in Brazil, ourselves in London. We really want to see the French become part of it and others. We've had conversations with all the Java User Group leaders and the intentions are all very positive. What we're now focusing on is giving more practical guidance on how they can go about running workshops and give feedback. That's the only thing really stopping this thing from taking off: how on earth do I run this and get the right feedback? JAX - Great to hear. Thanks gentlemen for chatting to us, and we'll see you at JAX London For more information about JAX London - the conference, speakers, keynotes and venue - visit the website. JAX London takes places October 15th-17th. Picture courtesy of MP4