Implementing HMAC authentication for REST API with Spring Security

Let's say you have to secure your REST API, but can't or don't want to use a secure connection (why u no use SSL/TLS!?!?), what you can do is implementing a HMAC mechanism to handle authentication. I assume you're well known about HMAC system; if not, read this great article from Riyad Kalla: Designing a Secure REST (Web) API without OAuth and this one on Amazon Web Service authentication system: Authenticating Requests Using the REST API.

HMAC in 6 (maybe 7) steps

For you lazy ones, here the HMAC mechanism summarized:

  1. Client and server share a secret access key and a public access key.
  2. Client create a request that contains three fundamental elements: the public key header (in plain text), a date header, a signature string calculated hashing some data of the request with the secret access key. This hash usually contains the http method, the uri path, the value of the date header (for reply attacks), all the content of the request (for POST and PUT methods) and the content type.
  3. Client send the request to the server.
  4. Server read the public key header and use it to retrieve the corresponding private access key.
  5. Server use the private access key to calculate the signature in the same way as the client did.
  6. Server check if the just-calculated signature matches with the one sent by the client.
  7. (OPTIONAL) To prevent reply attacks, server checks that the value in the date header is within an acceptable limit (usually between 5 and 15 minutes to account clock discrepancy). The value cannot be manipulated by malicious attacker because the date it's used as part of the signature. If someone change the date header, the server will calculated a different signature of that calculated by the client, so step 6 will fail.

Other than granting user identity (nobody should know the secret access key other than the client itself... and the server of course), this mechanism also ensure the integrity of the message. If someone change something in the request, the signature won't match.

Enough with the chit-chat. Let's start doing serious business.

First things first: the ingredients. That's what I've used:

Spring v3.2.3.RELEASE
Spring Security v3.1.4.RELEASE

That's all you need. Of course if you want to use a database to store your users (and of course you want) you have to include some ORM or JDBC or whatever... just add hibernate/jpa libraries.

The signature

The first thing to decide is how we want to compose the signature. Amazon offers some suggestion, so let's examine it:

StringToSign = HTTP-Verb + "\n" +
	Content-MD5 + "\n" +
	Content-Type + "\n" +
	Date + "\n" +
	CanonicalizedAmzHeaders +
	CanonicalizedResource;
Signature = Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) );
Authorization = "AWS" + " " + AWSAccessKeyId + ":" + Signature;

They use:

  • The http method.
  • The content of the request shrinked with MD5 hash.
  • The content-type.
  • The date (to prevent reply attacks).
  • Some sort of intricate gimmick to represent the URI.

Every one of these elements are separated by a new-line character. This whole mess is then passed to an algorithm that create the SHA-1 hash with the secret access key and returns our shiny signature.

Then they made an Authorization header similar to the basic authentication header, that is Basic <username>:<password>. Here instead there is AWS <public_access_key>:<signature>. We'll do something similar to this.

The filter

First we have to create a filter to intercept all requests that hit our secure service domain. For more informations about Spring Security filters, visit this page: The Security Filter Chain. The filter has to do just one thing: read the request and, based on the informations it contains, create an appropriate security token to send to the authentication manager. A security token is something that the authentication manager can understand and "rip apart" to handle the authentication process. For example, for a simple username/password authentication mechanism the token is just a POJO with two fields: an username and a password (you don't say!). Spring already provides such token.

Because our authentication mechanism check that the client calculated signature equals the server calculated signature, our token should contains:

  • the public access key or api key (aka username);
  • the client calculated signature;
  • the content used to calculate that signature, so that we can calculate our signature.

This is our RestAuthenticationToken (we have used the UsernamePasswordAuthenticationToken as a base class for convenience):

public class RestToken extends UsernamePasswordAuthenticationToken {

    private Date timestamp;

    // this constructor creates a non-authenticated token (see super-class)
    public RestToken(String principal, RestCredentials credentials, Date timestamp) {
        super(principal, credentials);
        this.timestamp = timestamp;
    }
	
    // this constructor creates an authenticated token (see super-class)
    public RestToken(String principal, RestCredentials credentials, Date timestamp, Collection authorities) {
        super(principal, credentials, authorities);
        this.timestamp = timestamp;
    }

    @Override
    public String getPrincipal() {
        return (String) super.getPrincipal();
    }
    
    @Override
    public RestCredentials getCredentials() {
        return (RestCredentials) super.getCredentials();
    }
    
    public Date getTimestamp() {
        return timestamp;
    }

}


public final class RestCredentials {

    private String requestData;
    private String signature;

    public RestCredentials(String requestData, String signature) {
        this.requestData = requestData;
        this.signature = signature;
    }

    public String getRequestData() {
        return requestData;
    }

    public String getSignature() {
        return signature;
    }

}

As you can see the token is just a field container. RestCredentials is just another container. It contains the content of the request to sign and the original client signature. So it behave like a password (ie something to check against something else).

So how do we create this token? Well, our filter must implements the Spring Filter interface (or extending an existing filter provided by Spring). This interface has the doFilter method that has the original http request as first parameter. So just read it, extract all data you need and simply create the token with it.

Wait a minute! If we have to read the content of the request to create the token, we can't read anymore when we have to process that content in our web service! When you read the content of a request, you access its InputStream object and that InputStream cannot be reset to its initial position to re-read the content of the request. It's simply not possible, you can't read the content twice.

But don't despair! We have a solution! An HttpServletRequestWrapper! A request wrapper is nothing more than a request object that store the content it read in an instance variable. This is an example of request wrapper (taken from this forum, but you can make your own if you like):

public class AuthenticationRequestWrapper extends HttpServletRequestWrapper {
     
    private final String payload;
    
    public AuthenticationRequestWrapper (HttpServletRequest request) throws AuthenticationException {
        super(request);
        
        // read the original payload into the payload variable
        StringBuilder stringBuilder = new StringBuilder();
        BufferedReader bufferedReader = null;
        try {
            // read the payload into the StringBuilder
            InputStream inputStream = request.getInputStream();
            if (inputStream != null) {
                bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
                char[] charBuffer = new char[128];
                int bytesRead = -1;
                while ((bytesRead = bufferedReader.read(charBuffer)) > 0) {
                    stringBuilder.append(charBuffer, 0, bytesRead);
                }
            } else {
                // make an empty string since there is no payload
                stringBuilder.append("");
            }
        } catch (IOException ex) {
            log.error("Error reading the request payload", ex);
            throw new AuthenticationException("Error reading the request payload", ex);
        } finally {
            if (bufferedReader != null) {
                try {
                    bufferedReader.close();
                } catch (IOException iox) {
                    // ignore
                }
            }
        }
        payload = stringBuilder.toString();
    }
 
    @Override
    public ServletInputStream getInputStream () throws IOException {
        final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(payload.getBytes());
        ServletInputStream inputStream = new ServletInputStream() {
            public int read () 
                throws IOException {
                return byteArrayInputStream.read();
            }
        };
        return inputStream;
    }

    public String getPayload() {
        return payload;
    }
}

As you can see, when the wrapper is created, it reads the request content and store its value in a string called payload. When another object call the getInputStream() method, it returns an InputStream of the payload string. Simple, yet effective.

So, now that we have our wrapper, we can read the content as much as we like. We can now create our token and send to the authentication manager without thoughts.

Following is an example of filter derived from the Spring GenericFilterBean (again, we use this as a base class for convenience, because it already handles all boring things like initiation and such):

public class RestSecurityFilter extends GenericFilterBean {

    // Enable Multi-Read for PUT and POST requests
    private static final Set<String> METHOD_HAS_CONTENT = new TreeSet<String>(String.CASE_INSENSITIVE_ORDER) {
        private static final long serialVersionUID = 1L; 
        { add("PUT"); add("POST"); }
    };
    
    private AuthenticationManager authenticationManager;
    private AuthenticationEntryPoint authenticationEntryPoint;
    private Md5PasswordEncoder md5;
    

    public RestSecurityFilter(AuthenticationManager authenticationManager) {
        this(authenticationManager, new RestAuthenticationEntryPoint());
        ((RestAuthenticationEntryPoint)this.authenticationEntryPoint).setRealmName("Secure realm");
    }

    public RestSecurityFilter(AuthenticationManager authenticationManager, AuthenticationEntryPoint authenticationEntryPoint) {
        this.authenticationManager = authenticationManager;
        this.authenticationEntryPoint = authenticationEntryPoint;
        this.md5 = new Md5PasswordEncoder();
    }

    @Override
    public void doFilter(ServletRequest req, ServletResponse resp, FilterChain chain) throws IOException, ServletException {
        // use wrapper to read multiple times the content
        AuthenticationRequestWrapper request = new AuthenticationRequestWrapper((HttpServletRequest) req);
        HttpServletResponse response = (HttpServletResponse) resp;
        
        // Get authorization header
        String credentials = request.getHeader("Authorization");

        // If there's not credentials return...
        if (credentials == null) {
            chain.doFilter(request, response);
            return;
        }

        // Authorization header is in the form <public_access_key>:<signature>
        String auth[] = credentials.split(":");
        
        // get md5 content and content-type if the request is POST or PUT method
        boolean hasContent = METHOD_HAS_CONTENT.contains(request.getMethod());
        String contentMd5 = hasContent ? md5.encodePassword(request.getPayload(), null) : "";
        String contentType = hasContent ? request.getContentType() : "";
        
        // get timestamp
        String timestamp = request.getHeader("Date");

        // calculate content to sign
        StringBuilder toSign = new StringBuilder();
        toSign.append(request.getMethod()).append("\n")
              .append(contentMd5).append("\n")
              .append(contentType).append("\n")
              .append(timestamp).append("\n")
              .append(request.getRequestURI());
        
        // a rest credential is composed by request data to sign and the signature
        RestCredentials restCredential = new RestCredentials(toSign.toString(), auth[1]);

        // calculate UTC time from timestamp (usually Date header is GMT but still...)
        Date date = null;
        try {
            date = DateUtils.parseDate(timestamp);
        } catch (DateParseException | IllegalArgumentException ex) {
            ex.printStackTrace();
        }
        
        // Create an authentication token
        Authentication authentication = new RestToken(auth[0], restCredential, date);

        try {
            // Request the authentication manager to authenticate the token (throws exception)
            Authentication successfulAuthentication = authenticationManager.authenticate(authentication);
            
            // Pass the successful token to the SecurityHolder where it can be
            // retrieved by this thread at any stage.
            SecurityContextHolder.getContext().setAuthentication(successfulAuthentication);
            // Continue with the Filters
            chain.doFilter(request, response);
        } catch (AuthenticationException authenticationException) {
            // If it fails clear this threads context and kick off the
            // authentication entry point process.
            SecurityContextHolder.clearContext();
            authenticationEntryPoint.commence(request, response, authenticationException);
        }
    }

}

The code is self-documented. We assume that the Authorization header is composed with the public access key and the signature, without any prefix like Basic or Digest or AWS. Who cares as long as it contains the credentials you need? Note that we have constructed the token using the constructor with three parameters. That constructor create a non-authenticated token (see the superclass APIs).

As I metion before, the doFilter() method wrap the original request around the AuthenticationRequestWrapper so that the content can be read multiple times. Then check if the method can have content (only POST and PUT method can have content) and in that case, calculate the request wrapper payload. If the request is a GET or DELETE, just use empty strings as content and content-type. Don't calculate the MD5 of an empty string! When the token is finally ready, we can send it to the authentication manager through the authenticate() method.

The authentication manager

The worst has passed. Now easy part. The authentication manager take the token, retrieve all the data it needs for creating the signature from it and check the new calculated signature against the client one.

The steps are:

  1. read the public access key or api key (aka the username) from the token;
  2. use that information to retrieve the secret access key from a provider (a database for example);
  3. calculate the signature with the content of the request and the secret key using the same procedure used by the client;
  4. check if the new calculated signature matches the one send by the client. If there's a match, access is granted, otherwise send a 401 response.

We made these steps in the authenticate method of our authentication manager:

@Component
public class RestAuthenticationProvider implements AuthenticationProvider {
    
    @Autowired
    private UserService service;
    
    public Authentication authenticate(Authentication authentication) throws AuthenticationException {
        RestToken restToken = (RestToken) authentication;

        // api key (aka username)
        String apiKey = restToken.getPrincipal();
        // hashed blob
        RestCredentials credentials = restToken.getCredentials();

        // get secret access key from api key
        String secret = service.loadSecretByUsername(apiKey);
        
        // if that username does not exist, throw exception
        if (secret == null) {
            throw new BadCredentialsException("Invalid username or password.");
        }
        
        // calculate the hmac of content with secret key
        String hmac = calculateHMAC(secret, credentials.getRequestData());
        // check if signatures match
        if (!credentials.getSignature().equals(hmac)) {
            throw new BadCredentialsException("Invalid username or password.");
        }

        // this constructor create a new fully authenticated token, with the "authenticated" flag set to true
        // we use null as to indicates that the user has no authorities. you can change it if you need to set some roles.
        restToken = new RestToken(user, credentials, restToken.getTimestamp(), null);
        
        return restToken;
    }

    public boolean supports(Class<?> authentication) {
        return RestToken.class.equals(authentication);
    }

    private String calculateHMAC(String secret, String data) {
        try {
            SecretKeySpec signingKey = new SecretKeySpec(secret.getBytes(), "HmacSHA256");
            Mac mac = Mac.getInstance("HmacSHA256");
            mac.init(signingKey);
            byte[] rawHmac = mac.doFinal(data.getBytes());
            String result = new String(Base64.encodeBase64(rawHmac));
            return result;
        } catch (GeneralSecurityException e) {
            throw new IllegalArgumentException();
        }
    }
}

Our authentication manager implements the AuthenticationProvider interface that requires two methods: authenticate() and supports(). The first is kinda obvious what do, the second tell Spring what kind of token this authentication manager can support.

The rest of the code is quite simple: we use a service that returns the secret from the username (the service call a DAO that access a database or another web service or LDAP, or whatever), we check if that username did return something (ie if the username exists), then we calculate the HMAC of the request data with the private method (our filter already compose the want-to-be signature in the correct pattern as described in the first paragraph about the signature). If the new signature doesn't match the client-one, then throw the BadGuy exception, otherwise return a new, fully authenticated token to be inserted in the security context holder. Note that we haven't assign any authority to the user. If you need to add some role to the user, just set a collection of authorities to in the constructor instead of null.

The authentication entry point

This is needed to send a 401 status code if the authentication fails It extends the BasicAuthenticationEntryPoint used for basic authentication and just set the response to 401. You can read more about it here.

public class RestAuthenticationEntryPoint extends BasicAuthenticationEntryPoint {
    
    @Override
    public void commence(HttpServletRequest request, HttpServletResponse response, AuthenticationException authException) throws IOException, ServletException {
        response.addHeader("WWW-Authenticate", "Basic realm=\"" + getRealmName() + "\"");
        response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
        PrintWriter writer = response.getWriter();
        writer.println("HTTP Status " + HttpServletResponse.SC_UNAUTHORIZED + " - " + authException.getMessage());
    }
    
}

The configuration file

It's time to tie all together with the spring security configuration xml. I assume you know what I'm talking about...

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
    xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/security
    http://www.springframework.org/schema/security/spring-security.xsd">
    
    
    <!-- Defines security domains -->
    <http create-session="stateless" entry-point-ref="authenticationEntryPoint" authentication-manager-ref="authenticationManager" use-expressions="true">
        <custom-filter ref="restFilter" position="FORM_LOGIN_FILTER" />
        <intercept-url pattern="/**" access="isAuthenticated()" />
    </http>
    
    <beans:bean id="authenticationEntryPoint" class="it.massimilianosciacco.hmac.spring.security.RestAuthenticationEntryPoint">
        <beans:property name="realmName" value="Secure realm" />
    </beans:bean>
 
    <beans:bean id="restFilter" class="it.massimilianosciacco.hmac.spring.security.filter.RestSecurityFilter">
        <beans:constructor-arg name="authenticationManager" ref="authenticationManager" />
    </beans:bean>
        
    <beans:bean id="restAuthenticationProvider" class="it.massimilianosciacco.hmac.spring.security.provider.RestAuthenticationProvider" />
 
    <authentication-manager alias="authenticationManager">
        <authentication-provider ref="restAuthenticationProvider" />
    </authentication-manager>
</beans:beans>

This is straight forward: we map all our domain with a security access that required authentication. Our authentication entry point is the RestAuthenticationEntryPoint, the filter is the RestSecurityFilter and the authentication provider is the RestAuthenticationProvider.

That's it. Deploy your application and cross your fingers. 9 out of 10 Spring will complain about some mis-configuration (often due to mispelling).

You can use a simple java client to test this out. Something like this:

public void testService() throws IOException {
    HttpPut request = new HttpPut(path);
    
    // content plain text
    String contentToEncode = "{\"items\":[{\"id\":1,\"value\":3},{\"id\":234,\"value\":8}]}";
    String contentType = "application/json";
    StringEntity data = new StringEntity(contentToEncode, contentType, HTTP.UTF_8);
    
    String date = DateUtils.formatDate(new Date());

    // create signature: method + content md5 + content-type + date + uri
    StringBuilder signature = new StringBuilder();
    signature.append(method).append("\n")
          .append(content).append("\n")
          .append(contentType).append("\n")
          .append(date).append("\n")
          .append(uriPath);

    request.addHeader(new BasicHeader("Date", date));
    String auth = username + ":" + RestUtil.calculateHMAC(password, signature.toString(), true);
    request.addHeader(new BasicHeader("Authorization", auth));

    // add data
    request.setEntity(data);
    
    // send request
    HttpClient client = new DefaultHttpClient();
    HttpResponse response = client.execute(request);
    
    int status = response.getStatusLine().getStatusCode();
    assert status == 200 : "Test failed";
}

You need the Apache http component library to use HttpPut. Here the maven signature:

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
</dependency>

 

Hope it helps.

Comments

Please provide the source code.

Thanks its working. only thing is missed csrf disable=true in xml.

Hi, Please provide source code.

Any idea why I'm getting: "The method getPayload() is undefined for the type AuthenticationRequestWrapper" I'm using Spring Security 3.2.7 and spring boot 1.2.5

Hi Ely.

Thank you for pointing out. The AuthenticationRequestWrapper is missing the getPayload() method. Just return the instance property. Something like this should work:

public String getPayload() {
    return payload;
}

Hi Paul,

Spring boot introduced some really cool features, but the security configuration is not one of them IMHO, so I never bother to learn all the APIs. Instead I rely on the @ImportResource and @Configuration annotation as explained here. Basically you have to create an empty class named SecurityConf for example, then put the two annotations. Something like this:

@Configuration
@ImportResource("classpath:securityConfig.xml")
public class SecurityConf {

}

I think you could also put it in the main class.

Hope it helps.