Wednesday, 24 February 2016

WildFly Elytron - SSL Configuration

One of the features being delivered by the upcoming WildFly Elytron integration within WildFly is the ability to provide SSL definitions centrally and then reference these definitions from elsewhere in the configuration.  This blog post demonstrates how to centrally configure SSL resources and subsequently use them to enable SSL both for web applications deployed to Undertow and for management of the server.  This blog post will also demonstrate how the same approach can be taken in domain mode to enable SSL to manage the server.

Environment

As WildFly 10 has been prepared for release the WildFly Elytron development has continued in parallel, to run the examples described here build the following three tags in order: -

https://github.com/wildfly-security-incubator/wildfly/tree/10.0.0.Final-Elytron1

The resulting server is the equivalent of a little before the WildFly 10 release but with the addition of the integration with WildFly Elytron.

After the WildFly build you will find a usable distribution in: -
 'wildfly/dist/target/wildfly-10.0.0.Final-Elytron1'.

Configuring SSL

For this example I am using a JKS KeyStore which contains a single entry which contains a private key and self signed certificate, certificate authority signed certificates would also be usable here.

To start up the application server with the WildFly Elytron subsystem included in the default configuration use the following command: -

bin/standalone.sh -c standalone-elytron.xml

Next start up the CLI and connect to the running server.

The approach taken with the WildFly Elytron subsystem is that different components required for security are defined independently of each other and then are assembled together for the final solution - this approach is taken to make it a lot easier to integrate custom implementations of the different components and to even allow other subsystems to advertise their own implementations within the application server so they can be used either with or in place of the Elytron implementations.

Within the CLI we next need to define three new resources within the Elytron subsystem: -
  1. KeyStore
  2. KeyManager(s)
  3. Server SSL Context
A new KeyStore can be added with the following command: -

./subsystem=elytron/keystore=server:add(type=jks, 
    path=/home/darranl/server.keystore, 
    password=keystore_password)

The arguments to this command should be fairly self explanatory.

The KeyManagers resource is then added with: -

./subsystem=elytron/key-managers=server:add(algorithm=SunX509, 
    keystore=server, 
    password=keystore_password)

In this command the 'keystore' argument is a reference to the KeyStore defined in the first command.

Then the Server SSL Context is defined: -

./subsystem=elytron/server-ssl-context=server:add(
    key-managers=server, 
    protocols=[TLSv1_2])

The 'key-managers' argument is a reference back to the KeyManagers resource defined in the previous command.

The server-ssl-context resource is where all policy information related to SSL will be defined, in this example however as this is a bare minimum definition we just reference the previously defined key-managers and specify that the protocol should be TLS 1.2.

Enable SSL With Undertow

Now that the resources are defined the next step is to enable the use of this resource within Undertow so the following command adds a https-listener referencing the previously defined server-ssl-context: -

./subsystem=undertow/server=default-server/
    https-listener=https:add(
        ssl-context=server, 
        socket-binding=https)

It should now be possible to connect a web browser to port 8443 and the previously defined SSL configuration within the Elytron subsystem will be used.

Enable SSL for Management over HTTP

Within the CLI with a couple more commands the same SSL resources can be used to enable SSL for access to the HTTP management interface: -

./core-service=management/management-interface=http-interface:
    write-attribute(
        name=ssl-context, 
        value=server)

The first command adds a reference to the previously defined server-ssl-context.

./core-service=management/management-interface=http-interface:
    write-attribute(
        name=secure-socket-binding, 
        value=management-https)

This second command then just enables HTTPS access on port 9993.

This last command will also require you to execute :reload for the changes to take effect.

Now you can connect to https://localhost:9993 and this will also be using the same previously defined SSL resources.

Enable SSL for Management over HTTP - Domain Mode

For this next step we now need to shut down the standalone server and start up domain mode with the host controller configured to enable Elytron.

bin/domain.sh --host-config=host-elytron.xml

When running within domain mode the host controller process can now contain it's own subsystems, first we need to repeat the three commands to add SSL resources but this time to the host controllers own Elytron subsystem.


./host=master//subsystem=elytron/keystore=server:
    add(
        type=jks, 
        path=/home/darranl/server.keystore, 
        password=keystore_password)

./host=master/subsystem=elytron/key-managers=server:
    add(
        algorithm=SunX509, 
        keystore=server, 
        password=keystore_password)

./host=master//subsystem=elytron/server-ssl-context=server:
    add(
        key-managers=server, 
        protocols=[TLSv1_2])

The only difference to the previous commands is in each of these the address has been prefixed with 'host=master' - this is so that the Elytron subsystem definition specific to the host controller is configured - apart from that it is exactly the same subsystem.

As before apart from a difference in address this SSL definition can be referenced in the same way: -

./host=master/core-service=management/
    management-interface=http-interface:
        write-attribute(
            name=ssl-context, 
            value=server)

How the port is specified in domain mode is slightly different so this is specified with: -

./host=master/core-service=management/
    management-interface=http-interface:
        write-attribute(
            name=secure-port, 
            value=9993)

The host controller can then be restarted with: -

./host=master:reload

At this point connections to https://localhost:9993 will be using this SSL definition.

Summary

The end result of this development will mean that administrators can learn how to define and tweak SSL policies all within the context of Elytron and the Elytron subsystem and then these policies can be applied across the application server where subsystems reference these resources - the aim being so that administrators do not need to learn how to configure SSL within each subsystem independently.

Advanced Configuration.

This blog post has focussed on the minimal steps required to get SSL enabled, I will follow up with some additional blog posts to cover the following features.

  • Alias Filtering
  • Cipher Suite Selection
  • Client Cert Authentication
  • Inspecting the KeyStore contents using the management model.
  • Inspecting (and invalidating) active sessions from the configured SSL resources.

Monday, 3 August 2015

WildFly Elytron 1.0.0.Alpha3 Released

At the end of last week we released WildFly Elytron 1.0.0.Alpha3

http://repo1.maven.org/maven2/org/wildfly/security/wildfly-elytron/1.0.0.Alpha3/

WildFly Elytron will be providing both SASL and HTTP authentication mechanisms for use within the application server, where possible stronger authentication mechanisms will be preferred over weaker mechanisms - there are challenges in achieving this but our ideal situation is to have no re-usable password representations transmitted over the network and also no passwords stored in reversible format on the local file system.

The 1.0.0.Alpha3 release has primarily been focussed on adding capabilities to the project to enable administrators to define policies to use when selecting SASL mechanisms and also some custom settings for when a mechanism is actually created.


  • Ability to specify protocol passed into the mechanism.
  • Ability to specify server-name passed into the mechanism.
  • Definition of properties to pass into mechanism for available mechanism evaluation and mechanism creation.
  • Filtering of mechanisms by name.
  • Filtering of mechanisms by name and provider.
  • Mechanism loading from a Provider[]
  • Service loader based mechanism loading from a provided ClassLoader.
  • Mechanism filtering based on supported credentials.


For the final item in that list at it's core the APIs that are being developers within WildFly Elytron are to enable a closer integration between the repositories that hold identity information and the authentication mechanisms.  A SecurityRealm is essentially the API used to interact with the repository of identities, from this we also obtain a RealmIdentity to represent an individual identity: -

As you can see from the above example both of these interfaces have a getCredentialSupport method, this method allows us to verify which credential types are supported in different situations and subsequently the mechanism filtering added in this release allows us to tailor the available authentication mechanisms to those actually compatible with the underlying realms.

In addition to the work on SASL based mechanisms selection the following Jira issues have also been resolved this release.

Release Notes - WildFly Elytron - Version 1.0.0.Alpha3


  • [ELY-247] - NullPointerException in SecurityProviderSaslServerFactory (Not all providers return services)
  • [ELY-227] - Add attribute mapping to Jdbc Security Realm
  • [ELY-175] - SASL mechanism availability should take into account credential support.
  • [ELY-189] - Support One-Time-Password SASL Mechanism
  • [ELY-197] - Add mechanism selection policies to the security domains.
  • [ELY-237] - Make it possible to configure the realm used on the client
  • [ELY-238] - Recognise realm callback on digest server, and make authorize callback come last
  • [ELY-239] - Make Anonumous auth set the anonymous principal name
  • [ELY-242] - Delegating SaslServerFactory supporting properties
  • [ELY-250] - Release WildFly Elytron 1.0.0.Alpha3
  • [ELY-33] - Identity Loading
  • [ELY-198] - Allow the association of Provider[] with a security domain and add support for mechanism loading.
  • [ELY-219] - Process subject roles in JaasAuthorizationIdentity
  • [ELY-223] - Add tests for the filesystem-backed realm
  • [ELY-244] - The properties map in the sasl factories should be Map<String, ?> not Map<String, Object>
  • [ELY-246] - Add github references to the pom

Next Release

The next release will be 1.0.0.Alpha4, the main feature to be included in that release will be the new APIs for HTTP authentication and an equivalent set of mechanism selection policies to allow an administrator control over HTTP authentication for their application server.

Wednesday, 8 July 2015

Upcomming WildFly Security Changes

For those of you that follow the WildFly development discussions you may be aware that we have a number of engineers currently engaged in a project called WildFly Elytron, for those not following essentially WildFly Elytron is a set of APIs and SPIs for application server security.  Although being developed to meet current requirements identified for the WildFly application server WildFly Elytron is a stand-alone library that could theoretically be used in other Java server environments.

The Need for Change

Historically application server security had been provided by making use of JAAS with the use of JAAS LoginModule implementations to verify a callers principal and credential combination, in the early stages of application servers this met the requirements fairly well - however in the modern age as increasing focus is given to security we find that this simple principal / credential pairing becomes harder and harder to use effectively.

As JBoss AS7 was developed our preference was to switch to stronger authentication mechanisms, for our native communication this meant a switch to SASL based authentication bound to the connection.  Due to incompatibility with JAAS this led us to a situation where we have two different security solutions within the application server, one focussed on SASL compatibility and one focussed on the legacy approach of principal / credential validation with integration between the two.

Having two different security solutions solving the same problem from completely different perspectives has let to a lot of confusion amongst users and from our perspective has left it difficult to provide new features - this has led to the initiation of the Elytron project to provide a single unified security solution across the whole application server.

Triangle of Compromises

Over the years working on security software within the application server I have come to the conclusion that whatever you do is always going to be bound by a triangle of compromises, enhance one node of the triangle and you risk reducing one or both of the other two.


Looking at a couple of examples.

Say you address how the passwords are stored, a common approach used here is a strong one way salted hash, if the storage is compromised at worst an attacker has an irreversible representation of the password that is next to useless to them without an expensive brute force search to attempt to identity the original password.  In the context of the triangle however the authentication mechanism you use for this password will now need to have dropped to plain text transmission, this may be countered buy using TLS on the connection but within the heap of the application server a plain text copy of the password will reside for some time.

Another option may be that some form of one time password is used instead, this improves the situation for both storage security and network security as the whole point of these passwords is that even if they are intercepted they can only be used once - the compromise however now is that usability is affected.

When an application server is deployed it is the role of the administrator to review these options in the context of their own installation, as an application server vendor it is not our role to prescribe the solution to be used instead our role is to provide the capabilities and options to allow the administrator to make these decisions and that is one of the main aims of the Elytron project.

What will Elytron Provide?

As mentioned above there are two predominant requirements of this project: -

  1. Provide a unified security solution across the application server.
  2. Give administrators the tools they need so that they can make decisions to configure their own environment effectively.
This is not an exhaustive list but here are some of the key areas we are looking to address: -
  1. Consistent use of terminology across the application server including consistent configuration across the server.
  2. Closer relationship between TLS configuration and authentication.
  3. A unified re-usable approach to configuration 
  4. Better client side control of the identity used to execute requests.
  5. Propagation of identities within different deployments in the same application server.
  6. Propagation of identities to a remote application server.

Underlying all of this is the emphasis on making sure stronger approaches to security are made available whilst simplifying the steps needed to achieve them.

What Next?

Following on from this will be a series of blog posts looking both at the APIs and SPIs of the Elytron project and how these will be represented in the application server in the form of the new subsystem currently being developed to expose this.

Related Links


Thursday, 27 November 2014

WildFly 9 - Kerberos Authentication with Remoting - Part 1

This blog post is in relation to a new feature added to WildFly 9 under WFCORE-106, however it is not currently included in a release so for now you will need to build WildFly yourself or use one of the nightly builds.

If you are not familiar with building WildFly have a look at HackingOnWildFly.

If you want to access a nightly build they are available from WildFly-latest-master.

Introduction

This blog post is the first part of a two part article on how to enable Keberos authentication for Remoting within WildFly 9, in Part 1 I discuss how to enable Keberos authentication for use with the CLI for managing the application server which makes use of JBoss Remoting to communicate with the server.

In Part 2 I will be looking at how to take an existing secured EJB deployment from within the quickstarts and enable Kerberos based authentication.

In a previous blog post WildFly 9 - Kerberos Authentication for Domain Management Over HTTP I have already described a lot of the steps required to enable Kerberos authentication for HTTP, for the steps that are also applicable for authentication with Remoting I will briefly mention them but will not duplicate the descriptions here.

Service Principals

One important aspect of Kerberos based authentication is that the service requiring authentication needs to be represented by it's own service principal, the naming of this principal is in the form '{service_name}/{hostname}' - where HTTP servers are secured using Kerberos the service_name is 'HTTP' - however for Remoting the service_name is expected to be 'remote'.

My Environment

My test environment that I am using for the basis of this blog post is the same as I described previously, the main difference here is that now I have a service principal called remote/web.dal.wildfly.org@DAL.WILDFLY.ORG which is in a keytab called remote.keytab.

As before any example commands will need to be updated to reflect your own environment.

Configuration Updates

As with the previous blog post system properties can be set as an easy way to define the realm name and the address of the KDC, after that similar updates also need to be made to the ManagementRealm: -


  1. Add a server identity for Kerberos.
  2. Add the keytab for the remote service principal.
  3. Add kerberos to the authentication resource of the security domain.
  4. Reload the server.
Steps 1, 3, and 4 are identical to those performed previously but in this example I would use the following command to add my remote keytab.


[standalone@web.dal.wildfly.org:9990 /] ./core-service=management/security-realm=ManagementRealm/server-identity=kerberos/keytab=remote\/web.dal.wildfly.org@DAL.WILDFLY.ORG:add(path=remote.keytab, relative-to=jboss.server.config.dir, debug=true)
{
    "outcome" => "success",
    "response-headers" => {
        "operation-requires-reload" => true,
        "process-state" => "reload-required"
    }
}


Running the CLI

As before I have used the kinit command to obtain my own ticket granting ticket from the KDC: -


[darranl@localhost bin]$ kinit darranl@DAL.WILDFLY.ORG
Password for darranl@DAL.WILDFLY.ORG: 
[darranl@localhost bin]$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: darranl@DAL.WILDFLY.ORG

Valid starting     Expires            Service principal
27/11/14 17:44:50  28/11/14 17:44:47  krbtgt/DAL.WILDFLY.ORG@DAL.WILDFLY.ORG
renew until 04/12/14 17:44:47


The next step is to start the CLI, as I start the CLI I set three system properties: -
  1. -Djava.security.krb5.realm=DAL.WILDFLY.ORG
  2. -Djava.security.krb5.kdc=kdc.dal.wildfly.org
  3. -Djavax.security.auth.useSubjectCredsOnly=false
The first two could be optional depending on local environmental settings but the third one is important as this allows the GSSAPI implementation to make use of the identity managed at the OS level.

In the case of my testing I am running the CLI on the same machine that I am running the application server so it is also important to ensure I disable local authentication otherwise Kerberos authentication will not get a chance to be used so I add the argument --no-local-auth.

This means the resulting command is: -

[darranl@localhost bin]$ ./jboss-cli.sh -c --controller=web.dal.wildfly.org \
    --no-local-auth -Djava.security.krb5.realm=DAL.WILDFLY.ORG \
    -Djava.security.krb5.kdc=kdc.dal.wildfly.org \
    -Djavax.security.auth.useSubjectCredsOnly=false [standalone@web.dal.wildfly.org:9990 /]

I can execute the :whoami command to double check the user I am authenticated as.


[standalone@web.dal.wildfly.org:9990 /] :whoami
{
    "outcome" => "success",
    "result" => {"identity" => {
        "username" => "darranl",
        "realm" => "ManagementRealm"
    }}
}

Additional Config

The example above is making use of a lot of default behaviour within the CLI, if you would prefer to control the Kerberos identity further it is possible to override the JAAS security domain used internally by adding the following system property to point to your own JAAS configuration -Djava.security.auth.login.config={config}, this configuration should contain a security domain called 'com.sun.security.jgss.initiate' that contains a Krb5LoginModule definition for your own environment. One example of a reason you may want to do this is if your clients identity can also be obtained from a keytab.






Friday, 17 October 2014

WildFly 9 - Kerberos Authentication for Domain Management Over HTTP

This blog post is in relation to a new feature added to WildFly 9 under WFCORE-105, however it is not currently included in a release so for now you will need to build WildFly yourself or use one of the nightly builds.

If you are not familiar with building WildFly have a look at HackingOnWildFly.

If you want to access a nightly build they are available from WildFly-latest-master.

Pre-Requisites

This blog post is specifically about the steps and configuration required to enable Kerberos based authentication for domain management over HTTP, this blog post does not describe how to define service principals and obtain the keytab or how to configure the web browser - there is already plenty of information out there covering those topics.

My Environment

The examples described in this blog post are all in the context of my test environment.

My KDC is service requests for the realm DAL.WILDFLY.ORG and the address of my KDC is kdc.dal.wildfly.org.

The HTTP management interface of my server installation is already listening on host web.dal.wildfly.org and I have a keytab called web.keytab which contains the service principal HTTP/web.dal.wildfly.org@DAL.WILDFLY.ORG.

Where you see any of these values in the following example obviously you will need to substitute in your own specific values for your own environment.

General Kerberos Settings

It is possible to rely on your local environments Kerberos configuration for things such as KDC identification and the resolution of the KDC's address, however for my own test environment where I am using a single KDC I prefer to just set a couple of system properties using the following CLI commands.

[standalone@web.dal.wildfly.org:9990 /] 
./system-property=java.security.krb5.realm:add(value=DAL.WILDFLY.ORG)
{"outcome" => "success"}
[standalone@web.dal.wildfly.org:9990 /] 
./system-property=java.security.krb5.kdc:add(value=kdc.dal.wildfly.org)
{"outcome" => "success"}

Enabling Kerberos Authentication

The steps to enable Kerberos authentication are split into two parts, first of all we need to define the servers own identity which is loaded from it's keytab and secondly once that is defined we need to enable the authentication.

Servers Identity

The first step is to now configure how the server will load it's own identity from a keytab, to do this I copy my web.keytab to the standalone/configuration folder of my installation and execute the following CLI commands: -

[standalone@web.dal.wildfly.org:9990 /] ./core-service=management/security-realm=ManagementRealm/server-identity=kerberos:add
{
    "outcome" => "success",
    "response-headers" => {
        "operation-requires-reload" => true,
        "process-state" => "reload-required"
    }
}
[standalone@web.dal.wildfly.org:9990 /] ./core-service=management/security-realm=ManagementRealm/server-identity=kerberos/keytab=HTTP\/web.dal.wildfly.org@DAL.WILDFLY.ORG:add(path=web.keytab, relative-to=jboss.server.config.dir, debug=true)
{
    "outcome" => "success",
    "response-headers" => {
        "operation-requires-reload" => true,
        "process-state" => "reload-required"
    }
}

The first command adds a container resource to the security realm to say that we will be adding a Kerberos identity, the subsequent command adds a reference to the keytab.  take note of where the principal name is specified within the address for the keytab and see how it needed to be escaped, also for this keytab I have enabled some additional debug output for when we communicate with the KDC.

Also note that these commands have put the server in the reload-required state.

It is possible that a single server could be accessible using multiple host names, for this reason it is possible to add multiple keytabs to the security realm,  the keytab definitions also have a for-hosts attribute that can be used to specify which keytab should be used for which host, failing that there will be an attempt to match the host name to the principal name.

In general we could proceed to the next section but it worth now executing :reload so that we can test our new configuration before enabling the actual authentication.

[standalone@web.dal.wildfly.org:9990 /] :reload                                  
                                                                                
{            
    "outcome" => "success",
    "result" => undefined
}

The keytab resource then has a test operation that can be invoked to verify the identity can be obtained.

[standalone@web.dal.wildfly.org:9990 /] ./core-service=management/security-realm=ManagementRealm/server-identity=kerberos/keytab=HTTP\/web.dal.wildfly.org@DAL.WILDFLY.ORG:test
{
    "outcome" => "success",
    "result" => {"subject" => "Subject:
Principal: HTTP/web.dal.wildfly.org@DAL.WILDFLY.ORG
Private Credential: /home/darranl/src/wildfly9/wildfly/dist/target/wildfly-9.0.0.Alpha2-SNAPSHOT/standalone/configuration/web.keytab
Private Credential: Kerberos Principal HTTP/web.dal.wildfly.org@DAL.WILDFLY.ORGKey Version 1key EncryptionKey: keyType=17 keyBytes (hex dump)=
0000: 8C D1 9C 24 86 84 D4 25   A3 12 FF DE A9 0D 9B 26  ...$...%.......&

At this point it is absolutely critical that you verify in the above output that one of the Private Credentials is actually a 'Kerberos Principal' and not just the reference to the keytab location.

Switch on Authentication

To switch on authentication a simple addition to the realm is required: -

[standalone@web.dal.wildfly.org:9990 /] ./core-service=management/security-realm=ManagementRealm/authentication=kerberos:add
{
    "outcome" => "success",
    "response-headers" => {
        "operation-requires-reload" => true,
        "process-state" => "reload-required"
    }
}

At this point :reload needs to be executed again and now I can try and connect using my web browser.

Load the Admin Console

Firstly I make sure I have obtained my own ticket granting ticket from the KDC: -

[darranl@localhost tmp]$ kinit darranl@DAL.WILDFLY.ORG
Password for darranl@DAL.WILDFLY.ORG: 
[darranl@localhost tmp]$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: darranl@DAL.WILDFLY.ORG

Valid starting     Expires            Service principal
17/10/14 13:08:18  18/10/14 13:08:15  krbtgt/DAL.WILDFLY.ORG@DAL.WILDFLY.ORG
renew until 24/10/14 13:08:15

As the security realm is based on the default configuration username/password based authentication is still enabled, for this reason you will need to ensure at least one user is defined by running the add-user script.

Now I load my web browser and attempt to connect using the admin console.


With no additional password prompt I am now authenticated as the user I previously obtained the ticket granting ticket for.

It is worth pointing out that my identity here includes the name of the realm, if desired when I added the authentication=kerberos to the realm I could have added the attribute remove-realm=true to strip the realm name from the username.

Finally if my web browser had not been enabled for Kerberos authentication or if I had not already obtained the ticket granting ticket I would have been prompted for a username and password as normal.

XML Configuration

Finally all of this configuration could have been defined manually in the application server configuration, the following is the resulting realm definition after the above CLI commands have been executed.

<security-realm name="ManagementRealm">
    <server-identities>
        <kerberos>
            <keytab principal="HTTP/web.dal.wildfly.org@DAL.WILDFLY.ORG" path="web.keytab" relative-to="jboss.server.config.dir" debug="true"/>
        </kerberos>
    </server-identities>
    <authentication>
        <local default-user="$local" skip-group-loading="true"/>
        <kerberos/>
        <properties path="mgmt-users.properties" relative-to="jboss.server.config.dir"/>
    </authentication>
    <authorization map-groups-to-roles="false">
        <properties path="mgmt-groups.properties" relative-to="jboss.server.config.dir"/>
    </authorization>
</security-realm>

Next Steps

The next steps now are to enable Kerberos authentication for our native interface so that clients such as the CLI and JMX Console can also take advantage of Kerberos based authentication.


Thursday, 11 September 2014

Kerberos EncryptedData NULL key / keytype mismatch.

For anyone who has ever been in the position of debugging Kerberos interoperability issues where Java is concerned I am fairly sure you would have come across the situation where you are attempting to search for some meaning in a cryptic error message you have received just to stumble on message posts from other users with the same error but no explanation as to the cause or a solution.

The purpose of this blog post is to describe one such error that I am currently investigating, I will say upfront however I have got to the bottom of why the error is being reported and have a possible workaround for some situations but the quest for a solution is ongoing.

If and when a complete solution is available I will update this blog post with the additional information.

In this case I have a JBoss EAP server handling incoming SPNEGO messages from different clients.

The Errors

Before describing the findings here are a couple of the errors I have been seeing: -

Caused by: KrbException: EncryptedData is encrypted using keytype NULL but decryption key is of type AES256 CTS mode with HMAC SHA1-96
at sun.security.krb5.EncryptedData.decrypt(EncryptedData.java:169) [rt.jar:1.7.0_17]
at sun.security.krb5.KrbCred.<init>(KrbCred.java:131) [rt.jar:1.7.0_17]
at sun.security.jgss.krb5.InitialToken$OverloadedChecksum.<init>(InitialToken.java:291) [rt.jar:1.7.0_17]
at sun.security.jgss.krb5.InitSecContextToken.<init>(InitSecContextToken.java:130) [rt.jar:1.7.0_17]
at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:771) [rt.jar:1.7.0_17]

And

Caused by: KrbException: EncryptedData is encrypted using keytype DES3 CBC mode with SHA1-KD but decryption key is of type NULL
at sun.security.krb5.EncryptedData.decrypt(EncryptedData.java:169) [rt.jar:1.7.0_17]
at sun.security.krb5.KrbCred.<init>(KrbCred.java:131) [rt.jar:1.7.0_17]
at sun.security.jgss.krb5.InitialToken$OverloadedChecksum.<init>(InitialToken.java:282) [rt.jar:1.7.0_17]
at sun.security.jgss.krb5.InitSecContextToken.<init>(InitSecContextToken.java:130) [rt.jar:1.7.0_17]
at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:771) [rt.jar:1.7.0_17]
... 35 more

For the purpose of identifying if your error message is related to this it is absolutely essential that you also compare the stack trace to ensure it is similar, i.e. the line numbers may be different but the classes in the stack should be almost identical.

The first of these error messages was reported when connecting to a server using Firefox from a Mac OS client, the second was reported using Firefox on a Fedora 20 client.

The Trigger

Within the SPNEGO authentication a Kerberos AP_REQ message is sent from the client to the server, within this message it is possible for the client to embed their own credential so that it can be delegated to the server.  For both of the error message above the KrbCred creation, EncryptedData decryption and the resulting validation and error are only applicable if the incoming message does contain a delegated credential.

When it comes to credential delegation and Kerberos this is something that the client decides it is going to do, the server has not actually requested that the client does delegate their credentials.

The Workaround

As this error is only encountered where credentials are delegated to the server the most simple workaround is to disable credential delegation in your client.  That way the AP_REQ message will not contain delegated credentials and the portion of the stack above reporting the error message will not be hit.

Unfortunately this workaround is only suitable if your server side service is not making use of delegated credentials but many services use SPNEGO without requiring delgation.

The Trigger II

So what is actually happening during the parsing that is triggering the error.

Within the Kerberos protocol a session key is established, certain fields can be encrypted using the session key and on parsing the message this session key can be used to decrypt the field.  

The encoded representation of an encrypted attribute contains a field that specifies the encryption type in use, for the above errors we see three types mentioned aes256-cts-hmac-sha1-96, des3-cbc-sha1, and also NULL.  Null is a special case as it specified that an encrypted field is not actually encrypted.

So back to the Java implementation, before parsing and possibly decrypting the delegated credential the Java implementation checks the type of the agreed session key.  If the session key is for aes256-cts-hmac-sha1-96, aes128-cts-hmac-sha1-96, or is arcfour then an assumption is made that the session key should be used to decrypt the credential.  For all other cases it assumes the encryption type will be NULL and that a NULL key should be used for the decryption.

So for the first error I posted above the Mac OS client used NULL for encrypting the delegated credential however as the session key is aes256-cts-hmac-sha1-96 the parsing is expecting this to have been used.

For the second error with the Linux client the session key is now des3-cbc-sha1, the Linux client did encrypt the delegated credential but the Java implementation has made the assumption it should not be doing that.

The Next Steps

As I said at the start at the time of posting this I do not have the solution, only the information that will hopefully make it easier for you to see what the error means and one possible workaround.

Overall this error is as a result of different approaches to message creation and parsing between the different clients and the Java implementation - so now I need to get to the bottom of which side is actually at fault.  My initial suspicion is the Java side as it should be possibly to identify the encryption type that was used before this error is reported but that is going to require some further investigation and discussions.








Thursday, 28 August 2014

WildFly 8 Security Realms - LDAP Caching

Another feature added to WildFly 8 that you may not be aware of unless you have gone looking for it is the ability to define caches on security realms where LDAP is being used for authentication or group loading.  This was added under WFLY-1523.

If you are familiar with the caching already in place for JAAS based security domains please be aware that the approach taken for the security realms is very different, for JAAS the key to the cache is essentially using the users supplied username and password and the cached value is the entire interaction with LDAP including all of the groups recursively found.  Within the security realms the caching is much more fine grained and it is the results for individual queries which are cached independently rather than the overall result cached as one.  This will be explained further in this post, for now this essentially means that it is possible that the results from queries for one user can be re-used for another user - this would occur where iteratively querying the group membership information of groups.

Base Configuration

Before describing how to enable caching it makes sense to start with an existing security realm already configured to authenticate users against LDAP and load their group membership information from LDAP, the following is one such possible configuration: -

    "core-service" : {
        "management" : {
            "security-realm" : {
                "LDAPRealm" : {
                    "authentication" : {"ldap" : {
                        "allow-empty-passwords" : false,
                        "base-dn" : "...",
                        "connection" : "MyLdapConnection",
                        "recursive" : false,
                        "user-dn" : "dn",
                        "username-attribute" : "uid",
                        "cache" : null
                    }},
                    "authorization" : {"ldap" : {
                        "connection" : "MyLdapConnection",
                        "group-search" : {"group-to-principal" : {
                            "base-dn" : "...",
                            "group-dn-attribute" : "dn",
                            "group-name" : "SIMPLE",
                            "group-name-attribute" : "uid",
                            "iterative" : true,
                          "principal-attribute" : "uniqueMember",
                            "search-by" : "DISTINGUISHED_NAME",
                            "cache" : null
                        }},
                        "username-to-dn" : {"username-filter" : {
                            "attribute" : "uid",
                            "base-dn" : "...",
                            "force" : false,
                            "recursive" : false,
                            "user-dn-attribute" : "dn",
                            "cache" : null
                        }}
                    }},
                }
            }
        }

This example has been cleaned up slightly but the main point to note is that this configuration has three key areas.

  • Authentication
If you are already using LDAP with security realms this should be the most familiar, effectively during authentication we discover the users distinguished name using this definition and attempt to connect to LDAP using their supplied credential to verify they are who they claim to be.
  • A username-to-dn definition in group search
When it comes to group searching we may rely on the availability of the users distinguished name, this block is not used in all situations but essentially it is a second attempt to discover a users distinguished name, this is more likely to be required if a second form of authentication was supported e.g. local authentication.
  • A group-to-principal group search

Then finally there is the group search definition, in this case it is an iterative search - what that means is that first all of the groups will be identified that the user is directly a member of - after that a search will be performed for each of those groups to identify the groups that those groups are a member of, this process continues until either a cyclic reference is detected or the final groups are not members of any further groups.

Before moving onto the next section please note in the above example there are three points that have a sub resource of "cache" : null , it is at each of these points that a cache can be defined to cache the LDAP interaction for that portion of the model.

The Caches

When enabling a cache it is possible to chose from one of two eviction strategies, the first being by access time and the second by search time.  If you choose by access time then the item in the cache will be evicted from the cache after a certain time period has elapsed since it was last used, if you choose by search time it will be evicted after the configured time has elapsed since the item was added to the cache regardless of if it has been accessed later.

In addition to choosing the eviction strategy the following attributes can also be set on a cache: -

  • eviction-time
This one should be fairly self explanatory and is the time in seconds used for evictions depending on the chosen strategy. 
  • cache-failures
This one is more important for the cache used in the authentication section and controls if the results of failed LDAP searches are cached to prevent the LDAP server from being hit again - this risk here is that without the next attribute a remote user could fill up the cache by trying many different users that do not exist.
  • max-cache-size
In addition to the time based strategy it is also possible to define a maximum size for the cache, if the maximum size of the cache is reached the oldest item in the cache will be evicted to make room for a new item being added - this is the final level of protection to prevent a cache that is caching failures from using up all of the available heap space to cache those failures.

Enabling The Cache 

So that is the description of the different configuration choices you can make, the following is the CLI command you can use to add a cache for the authentication related LDAP access: -

./core-service=management/security-realm=LDAPRealm/authentication=ldap/cache=by-access-time:add(eviction-time=300, cache-failures=true, max-cache-size=100)
This command enabled a cache that caches by access time with items automatically evicted after 5 minutes unless the maximum capacity of 100 items is reached - in addition to that failures are cached so the LDAP server will not be hit repeatedly for authentication attempts where a user is using an invalid user ID.


If the eviction by search time strategy was preferred this could have been achieved just as easily by using the following command: -
./core-service=management/security-realm=LDAPRealm/authentication=ldap/cache=by-search-time:add(eviction-time=300, cache-failures=true, max-cache-size=100)
I am not going to go through all of the other places in the model where caches can be enable but for the remaining two areas the command would be the same with just a modification to the address.

Cache Operations

Once caching is enabled the caches run fairly independently, however there are a couple of additional tasks you may want to perform yourself.  These examples are using the cache I defined above, however with address modifications they are applicable to the additional caches that can be defined.

Inspect The Cache Size

If you want to see how many entries are cached the following command can be executed to include runtime attributes: -

./core-service=management/security-realm=LDAPRealm/authentication=ldap/cache=by-access-time:read-resource(include-runtime=true)
{
    "outcome" => "success",
    "result" => {
        "cache-failures" => true,
        "cache-size" => 1,
        "eviction-time" => 300,
        "max-cache-size" => 100
    }
}
This was executed after I authenticated as one user so you can see one entry is in the cache.

Test The Cache Contents

If you want to check if the cache contains a reference to a specific user a command can be executed as follows: -
./core-service=management/security-realm=LDAPRealm/authentication=ldap/cache=by-access-time:contains(name=TestUserOne)
{
    "outcome" => "success",
    "result" => true
}
In this case TestUserOne is the user that I authenticated as.

Flushing The Cache

Finally it is possible to flush items from the cache, either the whole cache can be flushed or individual items can be flushed e.g.
./core-service=management/security-realm=LDAPRealm/authentication=ldap/cache=by-access-time:flush-cache(name=TestUserOne)
In this case the specific entry was flushed, had the name parameter been omitted from the operation the whole cache would have been flushed.

XML Configuration

That concludes this blog post, to complete this example here is the resulting XML for my security realm definition with the cache now defined: - 

<security-realm name="LDAPRealm">
    <authentication>
        <ldap connection="MyLdapConnection" 
              base-dn="ou=users,dc=group-to-principal,dc=wildfly,dc=org">
            <cache type="by-access-time" eviction-time="300" 
                   cache-failures="true" max-cache-size="100"/>
            <username-filter attribute="uid"/>
        </ldap>
    </authentication>
    <authorization>
        <ldap connection="MyLdapConnection">
            <username-to-dn>
               <username-filter attribute="uid"
                    base-dn="ou=users,dc=group-to-principal,dc=wildfly,dc=org" />
            </username-to-dn>
            <group-search group-name="SIMPLE" iterative="true" 
                          group-name-attribute="uid">
                <group-to-principal search-by="DISTINGUISHED_NAME" 
                     base-dn="ou=groups,dc=group-to-principal,dc=wildfly,dc=org" 
                     prefer-original-connection="true">
                    <membership-filter principal-attribute="uniqueMember"/>
                </group-to-principal>
            </group-search>
        </ldap>
    </authorization>
</security-realm>