Tag: kotlin

Creating DSL with Kotlin – Introducing a TLSLibrary

Creating DSL with Kotlin – Introducing a TLSLibrary

Creating DSL with Kotlin

Introducing a TLSLibrary

Motivation

If you’ve been following my recent posts about Kotlin, you’ve probably noticed me mentioning Domain Specific Languages (DSL) already. Kotlin as a programming language provides some powerful features that allow us to create those DSLs. One of these features, I also already introduced, is called Function Literals with Receiver, others are the invoke convention or infix notation.

In this post we’ll have a look at the concept of DSLs and certainly see how we can create one with Kotlin in a relatively simple example:
I’ve often been struggling with Java’s API for setting up SSL/TLS connections in scenarios where https communication is required for example. Just recently, I had to implement different kinds of these in one of our applications. Doing this, I once again felt like wanting to write a little library that can support me with that task, hiding away all the difficulties and of course the boilerplate.

Read More Read More

Please follow and like me 🙂
Kotlin Conventions – Operator Overloading

Kotlin Conventions – Operator Overloading

Operator Overloading and Conventions in Kotlin

Introduction

Kotlin supports a technique called conventions, everyone should be familiar with. For example, if you define a special method plus in your class, you can use the + operator by convention.
In this article I want to show you which conventions can be used and will provide some Kotlin code demonstrating the concepts of course.

Read More Read More

Please follow and like me 🙂
Kotlin Coroutines Guide

Kotlin Coroutines Guide

Approaching Kotlin Coroutines – An Extensive Feature Concurrent Programming in Kotlin

Introduction and Motivation

As already advertised on Twitter a few days ago, I was planning to run the rule over Kotlin Coroutines, which I actually did. Unfortunately, this took me longer than first expected because coroutines are huge, especially if you’re not yet familiar with the concept…​ Anyway, I’d like to share my gathered insights with you in the following post and hope to give a comprehensive overview.
Kotlin coroutines are definitely one of the “bigger features” as indicated by the following quote, taken from JetBrains’ blog:

We all know that blocking is bad under a high load, that polling is a no-go, and the world is becoming more and more push-based and asynchronous. Many languages (starting with C# in 2012) support asynchronous programming through dedicated language constructs such as async/await keywords. In Kotlin, we generalized this concept so that libraries can define their own versions of such constructs, and async is not a keyword, but simply a function.
This design allows for integration of different asynchronous APIs: futures/promises, callback-passing, etc. It is also general enough to express lazy generators (yield) and cover some other use cases.

To put it simply, coroutines have been introduced in order to provide simple means for concurrent programming. Probably many of you have already worked with Java, its Threads and concurrency classes; I myself did this quite a lot and actually I’m really convinced of its maturity.

Java Concurrency vs. Kotlin Coroutines

Tip
If you still catch yourself struggling with Threading and Concurrency in Java, I can recommend the book Java Concurrency in Practice to you.

Although Java’s solution is really well-engineered, it’s often difficult to utilise and (of course) very verbose. Another problem is, that Java doesn’t directly encourage non-blocking programming. You often find yourself starting threads without having in mind, that they’re very expensive and introduce blocking computations quickly (due to locks, sleeps, waits, etc.). Applying non-blocking patterns alternatively is really hard and error-prone.

Coroutines, on the other hand, are intended to be much easier and look like sequential code by hiding the complex stuff inside library functions. Yet, they provide a way to run asynchronous code without having to block threads, which provides new possibilities for applications [1]. Instead of blocking threads, computations are being suspended.
JetBrains tends to describe coroutines as “light-weight threads”; actually they are no Thread as we know them in Java. Compared to threads, coroutines are very cheap in creation and the overhead introduced by threads isn’t around. One reason is, that they’re not directly mapped to native threads. As you will see, coroutines are executed in Threads managed by the library.
An other important difference is limitation: Threads are limited because they rely on available native threads, coroutines on the other side are almost free and thousands can be started at once.

Concurrent Programming Style

Different styles of asynchronous/concurrent programming styles exist in various languages, which for example are: callback-based (JavaScript), future/promise-based (Java, JavaScript), async/await-based (C#) and so on. All these concepts can be implemented with coroutines because Kotlin doesn’t dictate any style initially. Instead, all concepts are already, or at least could in future, (be) implemented using coroutines.
As one additional benefit, as opposed to callback-based programming for example, coroutines promote a sequential kind of asynchronous programming, i.e. although your coroutines may execute multiple parallel computations, your code still looks sequential and therefore acquainted.

The Concept of Kotlin Coroutines

The term and concept “Coroutine” is anything but new. According to the Wikipedia article, it was created in 1958 already. Many modern programming languages provide native support: C#, Go, Python, Ruby, etc. The implementation of coroutines, also in Kotlin, is often based on so-called “Continuations”, which are “an abstract representation of the control state of a computer program”. We’ll capture that in How does it work – Implementation of Coroutines again.

Getting Started – The Basics

There’s a comprehensive tutorial available on https://kotlinlang.com that describes how a project needs to be set up in order to work with coroutines. Please have a look there or just check out my Kotlin_Examples repository on GitHub.

Kotlin Coroutine Ingredients

As already hinted, the Kotlin coroutine library provides an understandable high-level API that let’s us start quickly. One new modifier we need to learn is suspend, which is used to mark a method as “suspending”.
We’ll have a look at some easy examples using APIs from kotlinx.coroutines up next. But first, let’s learn what a suspending function is.

Suspending Functions

Coroutines rely on the keyword suspend, which is a modifier used in order to mark functions as “suspending” [2], i.e. that calls to such functions may suspend at any point. These can only be called from within coroutines, which in turn need at least one suspending function to be started.

suspend fun myMethod(p: String): Boolean {
    //...
}

As we can see in the example above, suspending functions look like regular functions plus the additional modifier. Keep in mind, that these methods can only be called from coroutines, other attempts will lead to compilation errors.

Coroutines can be thought of as a sequence of regular and suspending functions with an optional result being available after completion.

Hands-On

After all the blah blah I will now show some concrete coroutines in action. Let’s start with an example that shows the basics:

My first coroutine

fun main(args: Array<String>) = runBlocking { //(1)
    val job = launch(CommonPool) { //(2)
        val result = suspendingFunction() //(3)
        println("$result")
    }
    println("The result: ")
    job.join() //(4)
}
>> prints "The result: 5"

In this example, two functions, (1) runBlocking and (2) launch, are used, which are examples of coroutine builders. Many different builders exist, each of which starts a coroutine with different purposes: launch (fire and forget), async (promise returned), runBlocking (blocks thread) and so on.

The inner coroutine started by (2) launch does the actual work: a (3) suspending function is called which might suspend at any time, the result is printed after completion. The main thread, after starting the coroutine, prints a String before the coroutine finishes.
Coroutines started by launch return a Job immediately, which can be used for cancelling the computation or waiting for completion with (4) join() as we see here. Since calling join() may suspend, we need to wrap this call into another coroutine, which can often be achieved with runBlocking. This concrete coroutine builder (1)is designed to bridge regular blocking code to libraries that are written in suspending style, to be used in main functions and in tests” (Quoted API). If we removed the joining of the job, the program would stop before the coroutine can print the result.

Going deeper

A more lively example is the following: Imagine, you have to send an email in your application. Requesting the recipient address and rendering the message body are two very expensive tasks, which are independent of each other though. Being smart and using Kotlin, you want to make use of coroutines, performing both tasks in parallel, of course.
This is shown here:

email example async/await
suspend fun sendEmail(r: String, msg: String): Boolean { //(6)
    delay(2000)
    println("Sent '$msg' to $r")
    return true
}

suspend fun getReceiverAddressFromDatabase(): String { //(4)
    delay(1000)
    return "coroutine@kotlin.org"
}

suspend fun sendEmailSuspending(): Boolean {
    val msg = async(CommonPool) {             //(3)
        delay(500)
        "The message content"
    }
    val recipient = async(CommonPool) { 
        getReceiverAddressFromDatabase()  //(5)
    } 
    println("Waiting for email data")
    val sendStatus = async(CommonPool) {
        sendEmail(recipient.await(), msg.await()) //(7)
    }
    return sendStatus.await() //(8)
}

fun main(args: Array<String>) = runBlocking(CommonPool) { //(1)
    val job = launch(CommonPool) {
        sendEmailSuspending() //(2)
        println("Email sent successfully.")
    }
    job.join() //(9)
    println("Finished")
}

First, like already seen in the previous example, we use a (1) launch builder inside a runBlocking builder so that we can (9) wait for the coroutine’s completion. This isn’t new and neither is the (2) call to a suspending function sendEmailSuspending.
This method uses an (3) inner coroutine for getting the message content and (4) another suspend method getReceiverAddressFromDatabase for the address. Both tasks are executed in a separate coroutine build with (5) async. Note, that the calls to delay represent a non-blocking, coroutine suspending, alternative to Thread.sleep, which is used for mocking expensive computations here.

The async Coroutine Builder

The async builder is really simple and easy in its conception. As we know from many other languages, this method returns a promise, which is strictly speaking of type Deferredin Kotlin. All terms like promise, future, deferred or delay are often used interchangeably for describing the same concept: The async method promises to compute a value which we can wait for or request at any time.

We can see the “waiting” part of Kotlin’s Deferred objects in (7), where the suspending function (6) is called with the results of both prior computations. The method await() is called on instances of Deferred which suspends until the results become available [3]. The call to sendEmail is also wrapped in an async builder, for which completion we wait in (8) before returning its result.

What I didn’t explain: CoroutineContext

One important part of the above examples is the first parameter of the builder functions, which must be an instance of CoroutineContext. This context is what’s passed to a coroutine and provides access to the current Job (remember, this is what can be used outside a coroutine for cancellation or joining) for example.
Also, the current context may be utilised for launching inner coroutines, what has the effect, that the sub-coroutine’s Job is a child of its surrounding one. This again provides the possibility to cancel whole hierarchies of coroutines with a single cancellation request to the parent Job.
Different kinds of Elements are part of a CoroutineContext, one of which is CoroutineDispatcher.

In all examples shown, I used CommonPool, which is such a dispatcher. It makes sure, that the coroutines are executed in a thread pool managed by the framework. Alternatively, we could have used a confined thread, specially created, or implement our own pool for example. Contexts can even be combined easily using the overloaded + operator like so:

launch(CommonPool + CoroutineName("mycoroutine")){...}

Shared Mutable State

You may have thought something like the following during the last minutes: This whole coroutine stuff is really cool but what about synchronization, how do I share data between different coroutines?
Well, at least that’s what I was asking myself quite early because this seemed to be a reasonable question as most coroutines make use of thread pools (like CommonPool) to be dispatched in. It’s quite evident, that synchronization is just as important as we know it from Java for example. Actually, this is just about right: We can make use of acquainted strategies like using thread-safe data structures, confining execution to a single thread or using locks (see Mutex for further details).
Besides the common patterns, Kotlin coroutines encourage us to use a “share by communication” (see QA) style.

Concretely, an “actor” can be shared between coroutines. They can be used by coroutines, which may send/take messages to/from it. Let’s look at this in an example.

Actors

actor message passing example
sealed class CounterMsg {
    object IncCounter : CounterMsg() // one-way message to increment counter
    class GetCounter(val response: SendChannel<Int>) : CounterMsg() // a request with channel for reply.
}

fun counterActor() = actor<CounterMsg>(CommonPool) { //(1)
    var counter = 0 //(9) actor state, not shared
    for (msg in channel) { // handle incoming messages
        when (msg) {
            is CounterMsg.IncCounter -> counter++ //(4)
            is CounterMsg.GetCounter -> msg.response.send(counter) //(3)
        }
    }
}

suspend fun getCurrentCount(counter: ActorJob<CounterMsg>): Int { //(8)
    val response = Channel<Int>() //(2)
    counter.send(CounterMsg.GetCounter(response))
    val receive = response.receive()
    println("Counter = $receive")
    return receive
}

fun main(args: Array<String>) = runBlocking<Unit> {
    val counter = counterActor()

    launch(CommonPool) { //(5)
            while(getCurrentCount(counter) < 100){
                delay(100)
                println("sending IncCounter message")
                counter.send(CounterMsg.IncCounter) //(7)
            }
        }

    launch(CommonPool) { //(6)
        while ( getCurrentCount(counter) < 100) {
            delay(200)
        }
    }.join()
    counter.close() // shutdown the actor
}

This example shows the usage of an (1) Actor, which is a coroutine itself working on any context. The actor is holding the (9) relevant state of this sample application, which is counter. An other important feature, we haven’t considered so far, is a (2) Channel:


Channels

Channels provide a way to transfer a stream of values, similar to what we know as BlockingQueue (enables producer-consumer pattern) in Java but without any blocking methods. Instead, send and receive are suspending functions used for providing and consuming objects from the channel, implemented with FIFO strategy, making them fair.


The actor is, by default, associated to such a channel, which can be used in other coroutines (7) for sending messages to it. In the example, the actor iterates over the stream of messages from its channel (for works with suspending calls) handling them according to their type: (4) IncCounter messages make the actor change its state by incrementing the counter while (3) GetCounter makes the actor return its counter state by sending an independent message to the GetCounter‘s SendChannel.
The first coroutine (5) in main, just for the sake of convenience, launches a task which sends (7) IncCounter messages to the actor as long as the counter is less than 100. The second (6) just waits until the counter reaches 100. Both coroutines make use of  the suspending function (8) getCurrentCounter, which sends a GetCounter message to the actor and suspends by waiting on receive to return.

As we can see, the whole relevant state is confined to the specific actor coroutine. This solves the problem of shared mutable state.

More Features and Examples

If you really want to dive into coroutines and start working with it, I recommend the commonly known Kotlin documentation and especially want to suggest this fantastic guide.

How does it work – Implementation of Coroutines

I cannot go too much into detail here because this would exceed the post’s intention. I’m planning to write a follow-up with more detailed information on implementation, considering the generated byte code of coroutines as well, in the next weeks.
For now, let’s limit the following description to a “bird’s eye view”-ish one.

Coroutines do not rely on features of the operating system or the JVM. Instead, coroutines and suspend functions are transformed by the compiler producing a state machine capable of handling suspensions in general and passing around suspending coroutines keeping their state. This is enabled by Continuations, which are added as a parameter to each and every suspending function by the compiler; this technique is called “Continuation-passing style”.
If you can’t wait to know any details, you need to read this explanation.

Pro Tips, with Roman Elizarov

I’ve been talking to Roman Elizarov from JetBrains on Slack recently, who’s highly responsible for Kotlin’s coroutines 😉 Let me share the gathered information with you:

Q: The first question I had: When am I supposed to use coroutines and are there any use cases that still require threads to be used?

A: Rule of thumb by Roman:
Coroutines are for asynchronous tasks that wait for something most of the time. Threads are for CPU-intensive tasks.

Q: I mentioned, that the phrase “light-weight thread” sounds kind of inappropriate to me as it obscures the fact, that coroutines rely on threads since they are executed in a pool of threads. In my opinion, coroutines are rather a “task” being executed, stopped etc.

A: Roman answered, that the phrase “light-weight threads” is rather superficial, and that “coroutines are in many ways like threads from user’s standpoint.”

Q: As a last question, I wanted to know about synchronization. If coroutines are much alike threads, there must be the necessity of synchronizing shared state between different coroutines.

A: Roman told me, that known patterns of synchronization may be used, but it is recommended not to have any mutable shared state at all when we use coroutines. Instead, coroutines “encourage […​] to adopt “share by communication” style.

Conclusion

Coroutines once again demonstrate Kotlin’s magnificence. I wasn’t aware of coroutines until diving into the presented stuff, and was thinking, that Java’s concurrency tools are sufficient.

As opposed to Java, Kotlin encourages a totally different style of concurrent programming, which is non-blocking and naturally doesn’t make us start huge amounts of native threads.
In Java, it’s mostly normal to just start another thread or create new pools without having in mind, that this introduces a huge overhead and can even make our application slow due to blocking code for example. Coroutines, as an alternative, are said to be “light-weight threads”, which describes the fact, that they’re not mapped to native threads and therefore don’t drag along all the risks and problems we usually have to deal with (deadlocks, starvation e.g.). As we’ve seen, with coroutines, we normally don’t have to worry about blocking threads, synchronization is much more straight forward and not even necessary ideally as long as we pursue “share by communication”.

Coroutines also enable us to work with several different kinds of concurrent programming, each of which is either available in the library already (kotlinx.coroutine) or at least could be implemented easily.
Java developers in particular might most likely be acquainted to submitting tasks to a thread pool and waiting for results of futures then (ExecutorService), which we easily achieve by using async/await style. Yet, it’s not just an equal replacement, but a big improvement to what we know already.
Think about your concurrent Java code, all those checked exceptions, defensive locking strategies and a loooot of boilerplate code. With coroutines, it’s normal to write code sequentially by calling suspend functions, communicating with other coroutines, waiting for results, cancelling coroutines and more.

Perspective

Although I’m convinced that coroutines are truly fantastic, time will show whether it’s actually mature enough for highly concurrent applications. Many programmers will have to rethink and apply totally different concepts to their programs. I’m really curious about its future. As of now, coroutines are just experimental, which means JetBrains might adjust them in upcoming releases in order to adapt feedback from the community after playing around or even adopting coroutines in productive projects.

Well done! You seem to have read the whole article. I hope it was kind of useful. Please do not hesitate to get in touch, feedback’s always appreciated 🙂

 

Simon


1. Non-blocking programming became more and more popular in the last time because reactive programming gained influence.
2. Actually it’s “possibly suspending” since such a function might suspend at any point, but doens’t have to.
3. Keep in mind, that Java’s Future blocks a thread as soon as we call get() on it.

 

Please follow and like me 🙂
Function Literals with Receiver – Quick Introduction

Function Literals with Receiver – Quick Introduction

Function Literals with Receiver – The Foundation for DSLs and many Library Functions

Today I want to give a quick introduction to Function Literals with Receiver. This concept is what makes Kotlin great for designing Domain Specific Languages as we know it from Groovy for example. Also, Kotlins standard library includes many functions which many of you have already used or at least seen probably: apply and with are the most prominent ones.

Read More Read More

Please follow and like me 🙂
Kotlin on the JVM – Bytecode Generation

Kotlin on the JVM – Bytecode Generation

Kotlin on the JVM – How can it provide so many features?

Introduction

What exactly is a “JVM language”? Isn’t only Java meant to be run on the JVM?
Kotlin provides many features that aren’t available in Java such as a proper function type, extension functions or data classes. How was this even achievable? I’ve taken a deeper look at how Kotlin is made possible and what “JVM language” actually means. We’ll be having a look at Kotlin’s bytecode generation. I hope to help some people understanding a few things better 🙂

For a more detailed introduction to Kotlin’s features you can have a look at my recent posts like this one.

Please follow and like me 🙂
Sealed Classes in Kotlin – An Explanation

Sealed Classes in Kotlin – An Explanation

Sealed Classes in Kotlin – An Explanation

Today I came across Sealed Classes in Kotlin, which I had never heard of before. After some research I found that this concept is nothing new and is also available in Scala for example. So, yet another Scala feature JetBrains considered relevant and suitable for Kotlin? I like that 🙂 Read this post if you’re interested in more Kotlin features.

Actually this is a quite simple feature, which I’m going to explain in the following.

Feature Explanation

A sealed class can be subclassed and may include abstract methods which means sealed classes are abstract implicitly, although the documentation doesn’t clearly say so. To actually make a class “sealed” we have to put the sealed modifier before its name, as we can see here:

sealed class MyClass

Restriction

The important thing about sealed classes is that its subclasses must be declared in the same file as the sealed class itself.

Benefit

The feature allows us to define class hierarchies that are restricted in its’ types, i.e. subclasses. Since all subclasses need to be defined inside the file of the sealed class, there’s no chance of unknown subclasses which the compiler doesn’t know about.

Wait… Isn’t this what an enum actually is?

Kotlin’s sealed classes are some kind of extension of plain enums: As opposed to enums, subclasses of sealed classes can be instantiated multiple types and can actually contain state.

Use Case

The main advantage of sealed classes reveals itself if it’s used in when expressions. Let’s compare a normal class hierarchy to one of a sealed class handled in a when. First, we’ll create a hierarchy of Mammals and then put it in a method with a when:

open class Mammal(val name: String)
class Cat(val catName: String) : Mammal(catName)
class Human(val humanName: String, val job: String) : Mammal(humanName)
fun greetMammal(mammal: Mammal): String {
    when (mammal) {
        is Human -> return "Hello ${mammal.name}; You're working as a ${mammal.job}"
        is Cat -> return "Hello ${mammal.name}"
        else -> return "Hello unknown"
    }
}

The else is mandatory, otherwise the compiler will complain. This is because it just cannot verify that all possible cases, i.e. subclasses, are covered here. It may be possible that a subclass Dog is available at any time which is unknown at compile time.

Sealed “to the rescue”

But what if we knew there wouldn’t be other Mammals in our application? We’d want to leave out that else.

The problem of unknown subclasses can be avoided by sealed classes. Let’s modify the base class Mammal, its’ subclasses can remain the same.

sealed class Mammal(val name: String)

Now we can simply omit the else clause since the compiler can verify that all possible cases are covered because only the subclasses in the file of the sealed class exist, without exception. The method now looks as follows:

fun greetMammal(mammal: Mammal): String {
    when (mammal) {
        is Human -> return "Hello ${mammal.name}; You're working as a ${mammal.job}"
        is Cat -> return "Hello ${mammal.name}"
        // `else` clause not required, all the cases covered 
    }
}

That’s it. In conclusion really simple and handy, isn’t it? Have fun trying it yourself!

I’m going to create an example in KotlinByExample for this topic as soon as possible.

Finally, if you want to read about sealed classes in more detail I recommend the book Kotlin in Action to you!

Simon


 

Please follow and like me 🙂
Generic Types and Variance in Kotlin compared to Java

Generic Types and Variance in Kotlin compared to Java

Basics – What is Variance?

Many programming languages support the concept of subtyping, which allows us to implement hierarchies like “A Cat IS-An Animal“. In Java we can either use the extends keyword in order to change/expand behaviour of an existing class (inheritance) or use implements to provide implementations for an interface. According to Liskov’s substitution principle, every instance of a class A can be substituted by instances of its subtype B. The word variance, often referred to in mathematics as well, is used to describe how subtyping in complex aspects like method return types, type declarations, generic types or arrays relates to the direction of inheritance of the involved classes. There are three terms we need to take into account: Covariance, Contravariance and Invariance.

Read More Read More

Please follow and like me 🙂
Setup Vert.x Application written in Kotlin (Gradle Build)

Setup Vert.x Application written in Kotlin (Gradle Build)

I decided to write a Vert.x application in combination with Kotlin in a simple example because I’m really interested in Reactive Programming and love to use Kotlin. This post will give some basic information on Vert.x as a tool set for writing reactive applications on the JVM as well as a few words on Kotlin. In the end, I want to demonstrate how this application can be set up in Gradle. Of course, this will be just one out of a million possible solutions…​

Read More Read More

Please follow and like me 🙂

Enjoy this blog? Please spread the word :)