Shiro on App Engine

A demonstration which shows one way of integrating Shiro security with App Engine and Google Guice.

Front end user registration and password management is also provided in as minimalistic a fashion as we could manage

New! Sign in with Google or Facebook accounts.

App Engine and Shiro

We provide a Shiro Realm which works with the App Engine datastore via Objectify. Caching comes via App Engine's memcached service.

Guice and Shiro

Shiro's AOP features, specifically the method annotations, work with Guice. Otherwise Shiro configuration is done with the Shiro ini file.

Fork on Github

Get the code, file issues, etc. on the Github repository

GAEShiro on GitHub »

You can sign in from the link at the top right. There is a built-in account you can user, It has password pass. The zenith account is a normal user account.

You can also register for an account. You need to provide an Email which you control for this as a registration code will be sent to this Email address. Once you're registered you can use this account, unless someone suspends it.

If you forget your password you can reset it. An email is sent to you with a code and a link. Either enter the code or follow the link to do the reset.

For convenience we allow users to log in with Google or Facebook accounts. In each case we grab the Email address, but no registration is required. Its straightforward to add other OAuth 2 providers, in addition to Facebook. Note that the token is invalidated as soon as we've read the Email address. This increases security, but its an odd use of OAuth.

In practice all the URLs must run under HTTPS, since passwords are contained in the HTTP requests, and since we use Ajax calls where going from HTTP to HTTPS, which is cross-domain, is not allowed. This demo uses HTTPS throughout.

App Engine is a remarkable achievement. You can create a small free website and scale it indefinitely with almost no ongoing administration required. A wide range of useful services are available out of the box, with almost no set-up or maintenance needed.

With scalability comes a set of restrictions and limitations. Interfaces are non-standard: in particular you don't get SQL by default, which makes persistent storage and retrieval 'different'. Application instances are started on demand, so startup must be rapid. Startup is a particular challenge for Java which is known to initialize with the alacrity of a drowsy snail. App Engine charges for resources so you need to be careful to minimise their use and maximise caching.

There is a Google accounts service, but this can't be used in an application for a wider public who don't have or don't want use a Google login. Even with a user service many sites also need a permissions system to decide who gets to access what.

Shiro is a lightweight system for Authentication and authorization. Startup on App Engine for Shiro seems to be about 1 second (on top of other components of course), which is faster than for a heavier stack such as Spring Security. The shorter the startup the easier it is to scale an app by adding new instances in response to demand. So, Shiro is a good fit with App Engine and its worth making the adaptation to the Datastore and Memcached services.

If you're not using the built-in authentication system for Google email addresses then you'll likely want a basic system for user password management. Even if your preference is for a federated login you'll probably still need your own system for those users that don't want to use or can't use the federated system. This sample provides a basic password management system which can easily be extended, or used as-is

The fastest way to run App Engine is with basic servlets run from web.xml. This can be quite painful, so we're using Google Guice. Guice is a lightweight dependency injection framework with an extension for web applications and it makes wiring an application together much simpler.

Although Guice is considered to be lightweight (compared to Spring) it does slow down the startup of the application. This is because Guice does its wiring at startup, so pretty-much all your code will be loaded at once. With plain servlets you may be able to load classes incrementally.

This demonstration uses a simple trick to make the startup seem faster by using a static HTML page as the home page and performing an Ajax call from Javascript to do the initialization. Some crufty animation on load takes your attention away from the 5 second start-up delay! The startup page is generated from FreeMarker templates, just as the other pages, but at compile time, rather than run-time

The Shiro components which need to be adapted to App Engine are realms, the cache and the AOP-based annotations. If the annotations aren't required then they can be eliminated and startup time will be reduced.

To create a new realm only two methods need to be implemented, namely:

    AuthenticationInfo doGetAuthenticationInfo(AuthenticationToken token)
    AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection principals)

The only sort of token handle is the UsernamePasswordToken, so the principal and credentials are simple strings, in our case a user's email address and password. We use an email address as the user's identifier since we need it anyway to change passwords securely.

It is possible to set up users in the IniRealm and it makes sense to use this with our new realm -- the IniRealm is a good place to stash the administrator login for example. Our DatastoreRealm needs to be able to store an unlimited number of users persistently, so we need the datastore for this. Rather than using the raw datastore service we use Objectify which is a well designed and lightweight layer on top of the datastore, which just removes the rough edges without hiding the datastore structure.

Implementation of the DatastoreRealm is via a user object, GaeUser, which is keyed on the email address and has a single indexed field - the registration date. Using the email as the key means that lookup will be fast and cheap.

We have also implemented a Shiro-compatible cache, based on memcached. The cache expiry has been set to 5 minutes which should limit any consistency problems.

Since the DatastoreRealm is a singleton in each JVM, and since although memcached saves on datastore resources it is slow, we also include an in-memory cache in the realm, which is of limited size and evicts after 5 minutes, but is fast. A combination of in-memory and memcached caches is the best way to limit hits to the data store and to minimize the overall cost and the number of instances we need,

Another database object we use is a RegistrationString, which contains a one-shot code which is sent to users by email to allow them to (a) register and (b) change passwords. This code is time-limited so that archived emails don't cause a problem.

The final database object is a counter to keep track of the number of users we have. Its too inefficient to count users when the number is large so a counter is needed. We don't expect the counter to be changed very often (not more than once a second) so there's no need for fancy tricks like sharding.

Its a lot easier for users if they don't have to go through the hassle of registration, and remember a password for yet another application. So, we've set things up to allow Google and Facebook login. The implementation is reasonably generic and can easily be extended to other providers, such as Yahoo!, LinkedIn and so on. There is work involved I'm afraid as one needs to get credentials from each provider. Also, you are restricted as to the website on which you can use any particular set of credentials.

With Google if you're already logged in with the browser, you will find that you don't get the opportunity to choose the account you want to use -- you're logged in immediately with the account you came with. Similarly there is no way to force re-authentication.

We tried Google authentication with the built-in user service. This didn't work very well either -- same problems, so it makes sense to use OAUth. If you only need Google then we've left the code present and you wouldn't need to load the scribe library and hte OAuth adaption code.

Facebook login, which uses OAuth, words better. You always get to see the initial screen asking for permission, and you can force re-authentication.

We had to make some additions to get scribe to work with Google and OAuth2, so some work may be necessary for other providers. The parameters you need are not the same in each case, and the return JSON is not uniform so has to be adapted for each provider.

We're only considering services which provide a user email, on the principle that if some problem occurs with the provider we can still service these users by registering them in the normal way. Of course people change their Emails, but in this case its one at a time, not a whole class at once, and for our applications not a huge issue.

In an ideal world applications such as our would not need to do password management. Even though there are now a variety of identity providers, not all users will want to use them. So, as a fallback applications must provide traditional password management.

This sample provides the basics packaged so that anyone working with App Engine can incorporate it relatively simply, although presentational changes will be required.

In describing the flow we use relative URLs to describe processes controlled by servlets.

Control flow for registration

  1. Go to the /register.ftl URL. This loads a page with a registration form. The form contains an email field.
  2. The form is posted to /register. This is done using Ajax so we stay on the same page. An email message is sent to the user at the provided address. The email contains a code and a link. The user can either enter the code and the desired password on the page or click the link in the sent email and enter the code and password at the page which comes up. In either case we post or get to the /confirm URL with the code as a parameter.
  3. This checks the validity of the code, and if its valid adds a confirmation message to the current page. The user is logged in on the server and we are done.

Control flow for forgotten password

The control flow for a forgotten password is similar to that for registration with two slight differences.

  1. If the email address is already registered we don't bail out with an error message, as the presumption is that the person entering the data owns the email address.
  2. Some of the wording on the web pages is differed (Reset rather than Register). These differences can be expressed using the same template configured with slightly different arguments.

To provide a complete demo requires HTML pages. We're using Bootstrap as the CSS framework (it actually uses less to create CSS) which makes well laid out sites easy for those of us with no layout skills.

The HTML pages are organised using the Freemarker templating language. The index.html main page for example (this one) is pre-generated using Freemarker to avoid a wait while App Engine spins up an instance. This uses the Maven plugin for FMPP, the Freemarker pre-processor

OAuth for the social login is done with scribe which makes a complicated process incredibly simple. Heartily recommended.

All the administrative logic is done using jQuery's Ajax features. This is intended to decouple the login from the presentation if we wish to produce a mobile version of this demo, which will require different layouts. We also do some caching in JavaScript to minimise round trips to the server to determine login status. This doesn't compromise the server-side security but speeds things up and helps bring down App Engine's costs.


The com.cilogi.shiro.gae package should be easily re-usable. There are dependencies on Shiro, Objectify and Guava. The Guava dependency could be removed with a little effort, Objectify somewhat more.

The servlets in com.cilogi.shiro.web have parameters hard-wired and no I18N for strings, but the logic is re-usable.


Note that to use Facebook you need to register a Web App with Facebook and put the keys in src/main/resources/ The file will look something like this:

fb.local.apiKey=*your key here*
fb.local.apiSecret=*your secret here**your key here**your secret here*

We find it useful to register two apps, one to run locally in the dev server, and one to run in production. Hence the local and live keys.

The image of the lock is by renaissancechambara