Hamburger Icon

Ktor HTTP
Response and
Header Test Helpers

topher-lamey

Topher Lamey|July 6, 2021

Learn how to test an external cookie authentication case with Ktor and learn about some of our StackHawk helper functions.

At StackHawk, our application scanner HawkScan has to handle interactions with a variety of different applications and their security strategies. In order to successfully scan different applications, we have written many tests around various web authentication types. For cookie based authentication, this involves the setting up and handling of various HTTP requests and responses, including manipulating HTTP headers for setting and getting authentication cookies. Our scanner uses Kotlin and the excellent Ktor for our testing, along with JUnit 5. Recently, we had to test an external cookie authentication case and wanted to walk through how easy Ktor made it. Plus, we wanted to share some of our StackHawk helper functions.

For external cookie authentication, a request is first made to the application, which redirects to a different, "external" host and port. This external application then provides the cookie via the Set-cookie header.

Test Setup

Let's say we want to verify a basic happy path for external cookie auth in a test. This test would need to make the initial request against the app, follow the redirect to the external app, and then get the auth cookie from the header there. This is an integration test, so we want real servers, not mocks. In order to set this up, our test starts two embedded servers using our startTestWebApp function, which configures each server with a URL path, headers, and optional response body. We'll get into details of startTestWebapp further down, but this is all that's needed for our test to then make requests against the servers like our scanner does and verify the responses.

Note that the @AfterEach includes a stopTestWebApps() function that cleans up the embedded servers between each test (details further down).

Also note that in general, all our test helpers like startTestWebapp and stopTestWebapps() are defined in a TestUtils.kt file that's a shared test utility.

@Test
fun testExternalCookieAuth() {
    // given
    val authAppPort = startTestWebApp {
        routing {
            getBodyFixture("/login", "/auth/body_good.html", "/auth/header_200_cookie.txt")
        }
    }
    val appPort = startTestWebApp {
        routing {
            getHeaderFixture("/login", "/auth/header_302.txt", mapOf("REDIR_PORT" to authAppPort.toString()))
        }
    }
    // when/then/etc
 }
 
 @AfterEach
 fun teardown() {
   stopTestWebApps()
}

startTestWebApp - Start An Embedded Server

startTestWebApp starts an embedded server on a given host and port, and passes along an Ktor Application block to the server. Keep in mind that in our calling tests, this has a Application.routing block where we set up the URLs with their configuration for what we want returned.

It also adds the newly created server to a list so we can keep track of them.

val testWebApps: MutableMap<Int, ApplicationEngine> = mutableMapOf()

fun startTestWebApp(port: Int = findTcpPort(32768, 42768), host: String = "localhost", block: Application.() -> Unit): Int {
    val server = embeddedServer(Netty, port, host) {
        block()
    }
    testWebApps[port] = server
    server.start()
    return port
}

stopTestWebApps - Stop All The Embedded Servers

stopTestWebApps loops over our list of servers and shuts them down. In our test cases, this is called in the @AfterEach function.

fun stopTestWebApps() {
    testWebApps.entries.forEach {
        it.value.stop(1000, 1000)
    }
    testWebApps.clear()
}

getBodyFixture - GET With Response Body

getBodyFixture allows us to set up a GET request against a given URL path, the body returned, HTTP status code, custom headers, and the content type of the response. The body and headers (which also has the status code and content type) are defined in flat text files that are put in the project's test resources directory. In our test, we specify the relative path to the flat files and this will return them as part of the request.

fun Routing.getBodyFixture(
    path: String,
    bodyResourcePath: String,
    headerResourcePath: String? = null,
    contentType: ContentType = ContentType.Text.Html,
    overrides: Map<String, String> = emptyMap(),
    block: Routing.() -> Unit = {}
) {
    get(path) {
        val bodyText = withContext(Dispatchers.IO) {
            IOUtils.toString(
                TestUtils::class.java.getResourceAsStream(bodyResourcePath),
                Charset.defaultCharset()
            )
        }
        if (headerResourcePath != null) {
            val (status, headers) = getHeaderResource(headerResourcePath, overrides)
            call.addHeaders(headers)
            call.respondText(bodyText, status = HttpStatusCode.fromValue(status))
        } else {
            call.respondText(bodyText, contentType = contentType)
        }
    }
    block(this)
}

getHeaderFixture - A 302 Redirect GET Without A Body

getHeaderFixture lets us set up a URL path that returns only headers, which in the case of a 302 Redirect, is all that's needed. The headers (along with the status code and content type) are defined in a flat text file in our test resource path. The test just specifies that path and this will return its content as headers.

fun Routing.getHeaderFixture(
    path: String,
    headerResourcePath: String,
    overrides: Map<String, String> = emptyMap(),
    block: Routing.() -> Unit = {}
) {
    get("/") {
        call.respondText("test app")
    }
    get(path) {
        val (status, headers) = getHeaderResource(headerResourcePath, overrides)
        call.addHeaders(headers)
        call.respondHeadersFixture(headers, status)
    }
    block(this)
}

Header Resource File

The header resource files have a simple format. The first line is the returned HTTP status, and the rest are set as HTTP headers.

Runtime overrides can be denoted as tokens with @ on either side (like @REDIR_PORT@ below):

HTTP/1.1 302 Found

X-Frame-Options: SAMEORIGIN

X-XSS-Protection: 1; mode=block

X-Content-Type-Options: nosniff

Location: http://localhost:@REDIR_PORT@/users/2

Content-Type: text/html; charset=utf-8

Cache-Control: no-cache

X-Request-Id: 4f044df9-64a2-4f0c-a23c-84556d8fde57

X-Runtime: 0.069833

Transfer-Encoding: chunked

getHeaderResource - Read In Header Resource File (with optional token override values)

getHeaderResource reads in the header resource file and parses it for use in the response. For runtime values (like an embedded server's port), a map of token overrides can be provided to replace value holders in the file (like @REDIR_PORT@ above).

suspend fun getHeaderResource(
    headerResourcePath: String,
    overrides: Map<String, String> = mapOf()
): Pair<Int, List<Pair<String, String>>> {
    return withContext(Dispatchers.IO) {
        val rawHeaderLines = IOUtils.toString(
            TestUtils::class.java.getResourceAsStream(headerResourcePath),
            Charset.defaultCharset()
        ).lines().map { it.trim() }.filter { it.isNotEmpty() }
        val status = rawHeaderLines.first().split(" ")[1].toInt()
        status to rawHeaderLines.takeLast(rawHeaderLines.size - 1)
            .map {
                val parts = it.split(":", limit = 2)
                parts[0] to override(parts[1], overrides)
            }
    }
}

respondHeadersFixture - Sets Content Type and HTTP Status

private val engineOnlyHeaders = setOf("content-length", "content-type", "transfer-encoding")

suspend fun ApplicationCall.respondHeadersFixture(headers: List<Pair<String, String>>, status: Int) {
    val ct = headers.find { it.first.toLowerCase() == "content-type" }?.second
    respondText(ct?.let { it1 -> ContentType.parse(it1) }, HttpStatusCode.fromValue(status)) {
        ""
    }
}

addHeaders - Sets Headers On The Response

fun ApplicationCall.addHeaders(headers: List<Pair<String, String>>) {
    headers.forEach {
        if (!engineOnlyHeaders.contains(it.first.toLowerCase())) {
            response.header(it.first, it.second)
        }
    }
}

Verification

With all that in place, our test with a few lines of code, can then do the following sequence:

Issue a GET /login HTTP/1.1 against the appPort

Read the 302 Redirect response

Issue a GET /login HTTP/1.1 against the appAuthPort

Read the 200 OK response and verify the value in the Set-cookie header

Summary

For testing external cookie authentication, having control over HTTP responses and headers is needed. The test client needs to make a GET that returns a 302 to a dynamic port, then make another GET against the dynamic port, and finally get the cookie. Ktor provides a deep toolbox of hooks for setting things up, and we have written some helper wrapper functions to make our tests easy to write, read, and maintain.


Topher Lamey  |  July 6, 2021

Read More

Add AppSec to Your CircleCI Pipeline With the StackHawk Orb

Add AppSec to Your CircleCI Pipeline With the StackHawk Orb

Application Security is Broken. Here is How We Intend to Fix It.

Application Security is Broken. Here is How We Intend to Fix It.

Using StackHawk in GitLab Know Before You Go (Live)

Using StackHawk in GitLab Know Before You Go (Live)