As part of providing a REST API, we have the need to track certain API requests at a level higher than what raw HTTP logs in something like an ALB provides. In some cases, we want the ability to do additional lookups to augment the tracked data. In other cases, we want the ability to bring the tracked data into our applications.
Our APIs are written in Kotlin using SpringBoot, and after some research and design, we ended up with a custom annotation called @Auditable. This @Auditable annotation hooks into the HTTP request lifecycle via an Interceptor to provide a way to tag a controller method as being something that we want to track/audit. What we came up with provides the ability to automatically look for and record specific URL parameters and path values, and optionally add arbitrary data to the audit record if needed.
The Goal
Here's a slimmed down example of what we want the annotation use to look like. We have REST controller with a function that handles requests. The @Auditable annotation takes a type to mark the audit, and ideally the orgId URL parameter value would get passed along with the audit data as well. In reality, this would have other parameters or a RequestBody, but this is just meant to show the use of @Auditable.
@RestController
class API {
@RequestMapping("/{orgId}", method = [RequestMethod.POST])
@Auditable(type = AuditType.ORG_UPDATED)
fun createOrg(@PathVariable("orgId") orgId: String) {
// update the org
}
}Implementation
With that usage in mind, our implementation consists of first defining the annotation and its types. Then we use an Interceptor to put an audit payload object into the request attributes. The Interceptor also auto-populates the payload with any known URL/request params. After that, the Controller handles the request and can put data in the audit payload if needed. Once the request completes, the Interceptor grabs the data payload and invokes a custom @Service to persist the audit data.
Annotation and Type
First, let's define the annotation. Like we saw in the usage example above, all it needs is the type of audit.
@Target(AnnotationTarget.FUNCTION)
annotation class Auditable(
val type: AuditType
)The AuditType enum contains the list of audit event types. The types of audits are generally "write" focused, so we know when data changes. But we can also use this for tracking arbitrary events in the system too.
enum class AuditType {
ORG_CREATED,
ORG_UPDATED,
ORG_DELETED,
SCAN_CREATED,
SCAN_UPDATED,
SCAN_DELETED,
SCAN_ASSET_REQUESTED
}Payload
The AuditPayload stores audit info during the request's lifecycle.
class AuditPayload(
payload: Map<Type, String> = mapOf(),
private val pPayload: MutableMap<Type, String> = mutableMapOf()
) {
init {
pPayload.putAll(payload)
}
enum class Type(val label: String) {
ORG_ID(AuditableInterceptor.ORG_ID),
SCAN_ID(AuditableInterceptor.SCAN_ID),
SCAN_ASSET_FILENAME(AuditableInterceptor.SCAN_ASSET_FILENAME)
}
fun add(key: Type, value: String) {
pPayload[key] = value
}
fun get(key: Type) = pPayload[key]
}Interceptor
An Interceptor hooks into the controller's request lifecycle, determines if the handler is our @Auditable annotation, and if so, does audit processing logic.
Interceptor Context Config
Spring needs to know about custom Interceptors. Here's how ours gets wired into the Spring Context.
@Configuration
class WebMvcConfig(val auditableInterceptor: AuditableInterceptor) : WebMvcConfigurer {
override fun addInterceptors(registry: InterceptorRegistry) {
registry.addInterceptor(auditableInterceptor)
}
}Interceptor Class
The preHandle function lets us look for url and path params before, and puts them in a new request attribute object type called AuditPayload. Note this AuditPayload can be used for storing arbitrary data via the Controller too.
Then a postHandle function lets us do something with the AuditPayload once the request has been serviced by the Controller. In this case, we defined a service called AuditSendService that takes the audit data for its use.
@Component
class AuditableInterceptor(auditSendService: AuditSendService) : HandlerInterceptorAdapter() {
// List of URL or Path params to automatically put in an audit record
// NOTE: These need to be identical to what the controller uses for request mapping and path definitions
companion object {
const val ORG_ID = "orgId"
const val SCAN_ID = "scanId"
const val SCAN_ASSET_FILENAME = "scanAssetFilename"
const val AUDIT_PAYLOAD = "auditPayload"
val paramsToSearch =
listOf(ORG_ID, SCAN_ID)
}
// Adds an AuditPayload object auto-populated with known URL and path params to this request
override fun preHandle(request: HttpServletRequest, response: HttpServletResponse, handler: Any): Boolean {
if (handler is HandlerMethod) {
val annotation = handler.getMethodAnnotation(Auditable::class.java)
if (annotation != null) {
val payload = AuditPayload()
request.setAttribute(AUDIT_PAYLOAD, payload)
populatePayloadByParam(request, payload)
}
}
return true
}
// Takes the AuditPayload and sends it to the auditSendService
override fun postHandle(
request: HttpServletRequest,
response: HttpServletResponse,
handler: Any,
modelAndView: ModelAndView?
) {
if (handler is HandlerMethod) {
val annotation = handler.getMethodAnnotation(Auditable::class.java)
if (annotation != null) {
val payload = request.getAttribute(AUDIT_PAYLOAD) as AuditPayload
val auditDetails: AuditDetails = getAuditDetails(request.userPrincipal as OAuth2AuthenticationToken)
auditSendService.send(
auditId,
annotation.type,
auditDetails,
request.remoteAddr,
payload
)
} else {
logger.warn("Not sending audit - response code is ${response.status}")
}
}
}
// Pulls user details from the auth token
private fun getAuditDetails(token: OAuth2AuthenticationToken): AuditDetails {
return AuditDetails(
userEmail = token.principal.attributes["email"] as String,
userName = token.principal.attributes["name"] as String
)
}
// Adds URL or Path param values to the AuditPayload
private fun populatePayloadByParam(
request: HttpServletRequest,
payload: AuditPayload
) {
val paths =
request.getAttribute(HandlerMapping.URI_TEMPLATE_VARIABLES_ATTRIBUTE) as Map<String, String>
paramsToSearch.forEach { param ->
getValueFromParamOrPath(paths, param, request)?.let { value ->
payload.add(AuditPayload.Type.values().first { it.label == param }, value)
}
}
// potentially add lookups for related data and add to audit payload data as needed
}
private fun getValueFromParamOrPath(paths: Map<String, String>, paramName: String, request: HttpServletRequest): String? =
if (paths[paramName] != null) {
paths[paramName]
} else {
request.parameterMap[paramName]?.let { orgParams ->
orgParams[0] // Not handled: duplicated param values
}
}
data class AuditDetails(
val userEmail: String,
val userName: String
)
}Audit Send Service
This is just a skeleton of what could happen with the audit data in the postHandle. The audit info could be written to a DB, sent to a JMS queue, or whatever makes sense for the system in question.
@Service
class AuditSendService() {
companion object {
private val logger = LoggerFactory.getLogger(AuditSendService::class.java)
}
fun send(
auditRecordId: String = UUID.randomUUID().toString(),
auditType: AuditType,
auditDetails: AuditDetails,
userIPAddr: String,
payload: AuditPayload
) {
// write to DB or send to JMS or whatever
}
}Detailed Controller Usage
With all that set up, a Controller just puts the @Auditable annotation on a RequestMapping method, and audit data will show up in the AuditSendService.send method. Any URL or Path params will be included, along with anything the controller puts in the AuditPayload object.
@RestController("api/v1")
class API {
companion object {
// Using the same strings across the Controller and Interceptor is important
const val ORG_ID = AuditableInterceptor.ORG_ID
const val SCAN_ID = AuditableInterceptor.SCAN_ID
const val SCAN_ASSET_FILENAME = AuditableInterceptor.SCAN_ASSET_FILENAME
const val AUDIT_PAYLOAD = AuditableInterceptor.AUDIT_PAYLOAD
}
// The 'orgId' path param will be automatically included in the audit payload
@RequestMapping("/{$ORG_ID}", method = [RequestMethod.POST])
@Auditable(type = AuditType.ORG_MODIFIED)
fun updateOrganization(@PathVariable(ORG_ID) orgId: String) {
// be a controller
}
// The 'orgId' path param and the 'scanId' URL param both will be automatically included in the audit payload
@RequestMapping("/{$ORG_ID}/scan", method = [RequestMethod.POST])
@Auditable(type = AuditType.SCAN_CREATED)
fun createScan(@RequestParam(SCAN_ID) scanId: String,
@PathVariable(ORG_ID) orgId: String) {
// be a controller
}
// The 'orgId' and 'scanId' path params both will be automatically included in the audit payload,
// plus the auditPayload request param will be passed in so the controller can add whatever
// it wants to that
@RequestMapping("/{$ORG_ID}/scan/{$SCAN_ID}/asset", method = [RequestMethod.GET])
@Auditable(type = AuditType.SCAN_ASSET_REQUESTED)
fun createScan(@PathVariable(SCAN_ID) scanId: String,
@PathVariable(ORG_ID) orgId: String,
@RequestAttribute(AUDIT_PAYLOAD) auditPayload: AuditPayload) {
val scanAssetFilename = lookupScanAssetFilename(scanId)
auditPayload.add(SCAN_ASSET_FILENAME, scanAssetFilename)
// be a controller
}
}C'est La Fin
This high level audit strategy is easy to use on the Controller side, automatically looks and saves known URL/request parameter values, and allows Controllers to add extra data for audit events as needed. We used a Spring Interceptor to hook into the pre- and post-handling of a request to carry along an audit payload. That payload gets filled out with known URL/request params and whatever else the Controller wants to add. Then we have a custom Service that is the end point for the payload, and the service can do whatever's needed (JMS, DB, etc). Our actual implementation has more nuanced things, such as different audit types, but generally looks like what's in the article and has been working well. Hope you enjoyed this trip through audit land!
