J-Fall 2019: Nathan Perdijk – GraalVM in practice at the Dutch National Police
Full title: J-Fall 2019: Nathan Perdijk – GraalVM in practice at the Dutch National Police: running R from Scala in a Microservice architecture.
A lot of the best Data Science is done in R, but getting R to run as a streaming application in a complex environment using only Open Source tooling is an absolute nightmare. Making R interface with Kafka, or S3, or anything else, means giving up version control to external packages, while letting R talk to external services through a Docker container in networked VMs requires superhuman resolve. We were living through this nightmare. Our Data Scientists lying awake in the middle of the night, filled with Docker-induced-terror at the Kafka-esque transformation of their job. Their daily work rapidly morphing into “anything but” their core competency… It was time we found another path. Then a colleague mentioned he had heard something about integrating R into a Java application… Could this be our Salvation? Our Light in the Darkness? Our Holy GraalVM?
In this talk we will explore how we brought GraalVM into practice at the Dutch National Police by integrating R code into a Spring Boot 2 application written in Java and Scala. We will show code samples of the required wiring to get R to run on the JVM and discuss the various pitfalls and workarounds we discovered when putting this cutting edge tool to the test. We will discuss how GraalVM and FastR helped bring our Data Scientists back to the work they enjoy, while leaving the stream integration to languages and frameworks much better suited to the job.
