Im a big fan of consistency when it comes to API development and feel this is even more so when developing Serverless API’s using Lambda. I also hate to repeat the same operation and given the number of “functions” needed in a Lambda Serverless API it was necessary to create a common set of helper functions that I can just use and not have to think about.

The code I’m discussing can be found in a GitHub repository here.

When creating a somewhat complex API I’ve taken to using the proxy integration with API Gateway. Some would argue you should leverage the power of API Gateway to perform validations and other operations for you and while I largely agree, I also like to have a bit more control in the Lambda function.

Common Operations

There are a few candidates for common operations but I focused on a couple:

  • Sending a properly formatted response back to API Gateway in a consistent manner.
  • Using my own set of custom “Lambda” exception classes to communicate issues.
  • When testing the handler functions, I wanted a single test “event” object that can be altered based on input parameters.

Project Structure

The project structure looks something like this:

- PublicLambdaHelpers
  - lambda_helpers_pkg
    - __init__.py
    - lambda_errors.py
    - lambda_proxy_response.py
    - lambda_proxy_response_wrapper.py
    - test_proxy_event.py
  - tests
    - tests.py
  - LICENSE
  - README.md
  - requirements.txt
  - setup.py

Lambda Proxy Response Wrapper

The first thing I wanted to tackle was removing the burdon of formatting a response back to API Gateway. Every handler function needed to do this and I wanted to strip away the details and simply focus on doing one of two things:

  • send back an appropriate payload dictionary (or message) or
  • raise an error

I love wrapper functions so this was a no brainer:

def lambda_proxy_response_wrapper(success_status_code=200):
    def wrapper(func):
        @wraps(func)
        def decorated_view(*args, **kwargs):
            resp = LambdaProxyResponse()

            try:
                result = func(*args, **kwargs)
                resp.success = True
                resp.payload = result
                resp.status = success_status_code
            except ......
                # handle all exceptions here..

            return resp.make_response()

         return decorated_view

    return wrapper

All I now have to do is add the wrapper decoration to any lambda function:

@lambda_proxy_response_wrapper()
def request_handler(event, context):
    return {'foo': 'bar'}

And this would return the following to API Gateway:

{
    "statusCode": 200,
    "headers": {
        "Content-Type": "application/json"
    }
    "body": "{\"status\": 200,\"payload\": {\"foo\": \"bar\"}}"
}

Fantastic; no longer need to worry about the correct format needed.

Lambda Proxy Response

The Lambda Proxy Response object is there to actually generate the correct response based on the values supplied to it. In the final response API Gateway is looking for, the body must be a string so it handles the conversion of the payload accordingly.

Note: If you have any interaction with DynamoDB you’ll know that it uses Decimal as the underlying data type and json doesn’t play nice with that. So, the response object also parses the payload object and converts them back to either integers or floats before converting to a string output. I picked this up from a discussion here.

class LambdaProxyResponse:
    def __init__(self,
                 status: int = 200,
                 success: bool = True,
                 payload: dict = None,
                 message: str = None):
        self.success = success
        self.message = message
        self.payload = payload
        self.status = status
        self.error = None
        self.error_type = None
        self.error_traceback = None
        self.event = None

    def __repr__(self):
        return "ServiceResponse"

    def __str__(self):
        print("ServiceResponse")

    def make_response(self, include_stacktrace=False, include_event=False):
        resp = {
            "statusCode": self.status,
            "headers": {
                "Content-Type": 'application/json'
            }
        }

        body = {
            "status": self.status
        }

        if self.message:
            body["message"] = self.message

        if self.payload:
            body["payload"] = replace_decimals(self.payload)

        if self.error:
            body["error"] = self.error

        if self.error_type:
            body["error_type"] = self.error_type

        if include_stacktrace and self.error_traceback:
            body["error_traceback"] = self.error_traceback

        if include_event and self.event:
            body["event"] = self.event

        resp["body"] = json.dumps(body)

        return resp

    def make_logging_response(self):
        return json.dumps(self.make_response(include_stacktrace=True,
                                             include_event=True))

I can also send in the original event object and/or a stacktrace in the event of an unhandled exception. These are useful when trying to debug.

Errors

I wanted a consistent set of errors that I can raise from within the my lambda functions and know they’ll be handled correctly.

  • NotFoundError
    • when the item requested is not found
  • MissingParameterError
    • a required parameter has been omitted
  • InvalidParameterError
    • an invalid parameter has been provided
  • AlreadyExistsError
    • when an item being added (to whatever) already exists and shouldn’t be overwritten.
  • RelatedRecordsExistError
    • if an object is being deleted but related items exist. User should remove the related items before attempting a delete on the parent.
  • ValidationError
    • when a validation attempt fails
  • InvalidFunctionRequestError
    • if the handler method is unable to determine what function to call
  • InvalidUserError
    • if the user cannot be found or expected user attributes are not part of the request event
  • GeneralError
    • alias to the base Exception object but you can include the event object for debugging

Going back to the wrapper function, I have an except clause for each of these so I can handle the error in whatever way I need. For now it’s a little repeated as I’m doing the same thing for most of them but provides me some flexibility if I ever needed something different.

except lambda_errors.GeneralError as e:
    resp.error_type = repr(e)
    resp.error = str(e.message)
    resp.status = e.status_code
    resp.event = e.event

except urllib3.exceptions.HTTPError as e:
    exc_type, exc_value, exc_traceback = sys.exc_info()
    resp.error_traceback = traceback.format_exc().splitlines()
    resp.error_type = repr(type(e))
    resp.error = str(exc_value)
    resp.status = 500
    logger.error(resp.make_logging_response())

except ValueError as e:
    exc_type, exc_value, exc_traceback = sys.exc_info()
    resp.error_traceback = traceback.format_exc().splitlines()
    resp.error_type = repr(type(e))
    resp.error = str(exc_value)
    resp.status = 500
    logger.error(resp.make_logging_response())

except Exception as e:
    exc_type, exc_value, exc_traceback = sys.exc_info()
    resp.error_traceback = traceback.format_exc().splitlines()
    resp.error_type = str(type(e))
    resp.error = str(exc_value)
    resp.status = 500
    logger.error(resp.make_logging_response())

For most of the custom errors, it’s handled in the same way as the GeneralError above; insert the error type, message and status (and the event for this one) into the response object so it’s sent back to the caller. As I mentioned you want to avoid including the event object but for debugging purposes this can sometimes be really helpful.

I cover “unhandled” exceptions a little differently. For these I’ll generate the stacktrace and include that in the response. However, this will only be included in the logging response, not the final response that’s sent back to the caller (for obvious reasons).

Conclusion

I’ve found this common code very helpful when developing Serverless applications by removing the required mundane response formatting allowing me to focus on the necessary logic. Having specific errors that cover the various issues means I’m consistent across all my API’s in how I report problems.

Leave a comment