Transitioning To A More Open Technology Stack

Snowy PostboxI’m currently working with some large Java monoliths which talk to each other over ActiveMQ. There are several aspects of the architecture that I’d like to change. Certainly, new production environments (Kubernetes, etc) mean that monoliths are not required because of the overhead of deployment, and the benefits of easier testing and more modular architecture mean that I think the expense of migrating to smaller services will be well worth it. With such an established code base though, the question I’m grappling with is how can we transition to a better, more open technology stack without needing to rewrite from scratch and do a big bang deployment.

Currently I’m toying with the idea of writing an ActiveMQ to Web Sockets bridge.  Web Sockets are a way of emulating a direct TCP connection in a web browser, although a more normal use case is to send and receive a stream of JSON encoded events. Although Web Sockets were created for use in browsers all languages have libraries available which will allow you to connect to a server.

ActiveMQ natively supports connecting over Web Sockets, so why would I propose building a bridge application? In our case the messages being exchanged are binary encoded, so you can’t decode them unless you’re running Java and have the same library used to send the messages. By building an application to act as a bridge you get much more control over the Web Socket API than if you use the native ActiveMQ implementation, so you can tidy up the JSON representations you use and easily make any other improvements to the API that you want.

Spring is our current Java Framework of choice, which conveniently has a built-in HTTP server which supports Web Sockets. Combining that with our shared library for connecting to ActiveMQ results in a Web Socket server in just a couple of hundred lines of code, and most of that is actually converting the message objects into a nice JSON representation.

In future posts I’ll talk about our progress migrating to a more open environment, but first let’s go through how to build the bridge. I’ve chosen a simple REST API.

  • GET /topic will return a list of topics.
  • GET /topic/{topic} returns a single message from the topic (not much use in reality, but useful for testing).
  • CONNECT /topic/{topic} opens a web socket connection to a topic, which lets you send and receive a stream of events.

The first step is to enable web sockets on the right URL.

@Configuration
@EnableWebSocket
public class WebSocketConfig implements WebSocketConfigurer {
    @Autowired
    private SocketHandler sockerHandler;

    public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
        registry.addHandler(sockerHandler, "/topic/{topic}")
            .setAllowedOrigins("*");
    }
}

Next up we set up the normal HTTP end points. Here I’m using two objects to manage the ActiveMQ connections and JSON serialisation/deserialisation. If like us you have shared libraries to do your messaging for you then you can just plug those in, and there are some many JSON serialisers you can just pick your favourite.

A key thing with this class is to specify the method of the requests so we can use the same URL as we registered for the web sockets without clashing.

@Controller
@RequestMapping("/topic")
public class TopicHandler {
    @Autowired
    private JmsConnectionManager jmsConnectionManager;

    @Autowired
    private JsonSerialiser jsonSerialiser;

    @RequestMapping(method = RequestMethod.GET)
    public @ResponseBody List GetTopics() {
        return jsonSerialiser.serialise(jmsConnectionManager.getTopics());
    }

    @RequestMapping(value="/{topic}", method = RequestMethod.GET, headers = "Connection!=Upgrade")
    public @ResponseBody String GetTopic(@PathVariable("topic") String topic) {
        ActiveMqTopicController controller = jmsConnectionManager.getTopicController(topic);

        return jsonSerialiser.serialise(controller.getMessage(), BaseMessage.class);
    }
}

Lastly, we handle the web socket connections. There are three methods of TextWebSocketHandler that we need to override. handleTextMessage is called when a message is received from the client, while afterConnectionEstablished and afterConnectionClosed are called at the start and end of the connection. When the connection is established you need to connect to the JMS topic, and start streaming events.

@Component
public class SocketHandler extends TextWebSocketHandler {
    @Autowired
    private JmsConnectionManager jmsConnectionManager;

    @Autowired
    private JsonSerialiser jsonSerialiser;

    public SocketHandler() {
    }

    @Override
    public void handleTextMessage(WebSocketSession session, TextMessage message)
            throws InterruptedException {
        BaseMessage jmsMessage = jsonSerialiser.deserialise(message.getPayload(), BaseMessage.class);

        ActiveMqTopicController tc = jmsConnectionManager.getTopicController(getTopic(session));
        tc.publishMessage(jmsMessage);
    }

    @Override
    public void afterConnectionEstablished(WebSocketSession session) throws Exception {
        ActiveMqTopicController tc = jmsConnectionManager.getTopicController(getTopic(session));
        tc.addListener(session);
    }

    @Override
    public void afterConnectionClosed(WebSocketSession session, CloseStatus closeStatus) {
        ActiveMqTopicController tc = jmsConnectionManager.getTopicController(getTopic(session));
        tc.removeListener(session);
    }

    private String getTopic(WebSocketSession session) {
        String path = session.getUri().getRawPath();

        String[] components = path.split("/");

        return components[components.length - 1];
    }
}

With this fairly simple code in place, it’s dead easy to start integrating other languages, or single page apps running in a web browser into your previously closed messaged based system.


Photo of Snowy Postbox by Gordon Fu.

Advertisements

Introducing A New Language

code.close()At work, there is a discussion going on at the moment about introducing Kotlin into our tech stack. We’re a JVM based team, with the majority of our code written in Java and few apps in Scala. I don’t intend to discuss the pros and cons of any particular language in this post, as I don’t have enough experience of them to decide yet (more on that to come as the discussion evolves). Instead, I wanted to talk about how you can decide when to introduce a new language.

Programmers, myself included, have a habit of being attracted to anything new and shiny. That might be a new library, a new framework or a new language. Whatever it is, the hype will suggest that you can do more, with less code and fewer bugs. The reality often turns out to be a little different, and by the time you have implemented a substantial production system then you’ve probably pushed up against the limits, and found areas where it’s hard to do what you want, or where there are bugs or reliability problems. It’s only natural to look for better tools that can make your life easier.

If you maintain a large, long-lived code base then introducing anything new is something that has to be considered carefully. This is particularly true of a new language. While a new library or framework can have its own learning curve, a new language means the team has to relearn how to do the fundamentals from scratch. A new language brings with it a new set of idioms, styles and best practices. That kind of knowledge is built up by a team over many years, and is very expensive both in time and mistakes to relearn.

Clearly, if you need to start writing code in a radically different environment then you’ll need to pick a new language. If like us, you mostly write Java server applications and you want to start writing modern web-based frontends to your applications then you need to choose to add Javascript, or one of the many Javascript based languages, into your tech stack.

The discussion that we’re having about Java, Scala and Kotlin is nowhere near as clear-cut however. Fundamentally choosing one over the other wouldn’t let us write a new type of app that we couldn’t write before, because they all run in the same environment. Scala is functional, which is a substantial change in idiom, while Kotlin is a more traditional object-orientated language, but considerably more concise than Java.

To help decide it makes sense to write a new application in the potential new language, or perhaps rewrite an existing application. Only with some personal experience can you hope to make a decision that’s not just based on hype, or other people’s experiences. The key is treat this code as a throw-away exercise. If you commit to putting the new app into production, then you’re not investigating the language, you’re commiting to add it to your tech stack before you’ve investigated it.

As well as the technical merits, you should also look into the training requirements for the team. Hopefully there are good online tutorials, or training courses available for your potential technology, but these will need to be collated and shared, and everyone given time to complete them. If you’re switching languages then you can’t afford to leave anyone behind, so training for the entire team is essential.

Whatever you feel is the best language to choose, you need to be bold and decisive in your decision making. If you decide to use a new language for an existing environment then you need to commit to not only writing all new code in it, but to also fairly quickly port all your existing code over as well. Having multiple solutions to the same problem (be it the language you write your server-side, or browser-side apps in, or a library or framework) create massive amounts of duplicated code, duplicated effort and expensive context switching for developers.

Time and again I’ve seen introducing the new shiny solution create a mountain of technical debt because old code is not ported to the new solution, but instead gets left behind in the vague hope that one day it will get updated. New technology and ways of working can have a huge benefit, but never underestimate the cost, and importance, of going all the way.


Photo of code.close() by Ruiwen Chua.