Crux for GT Developers

Crux for GT Developers


Java WS Core component of the Globus Toolkit, built on Axis 1.x, provides a Web Service Development Kit (WSDK) for service developers to build Web Services Resource Framework(WSRF) compliant services. The component also provides common services that can be leveraged by services built on Java WS Core. Security support for Java Web Services developed using Globus Toolkit includes authentication, authorization, security policy enforcement and delegation. The security infrastructure builds on Axis 1.x security and other libraries such as, WSS4J and PureTLS, to provide support for Grid Security Infrastructure (GSI),

As GT has aged, technologies have improved and, as a result, some of the productivity gains provided by GT have diminished. Herein, we propose an evolution of GT taking advantage of these new technologies to improve the productivity of service developers and administrators and, in doing so, create a toolkit that is more flexible and sustainable. This paper motivates the next generation of Globus solution, Crux Toolkit, that builds on more recent WSDKs, and provides a collection of tools and services designed to simplify the process of developing secure Service-Oriented Computing. The Crux Toolkit will provide features that fall outside the scope of these WSDKs, providing features required to support the user stories.

This document describes a subset of features provided by Crux Toolkit, that is targeted at service developers who are currently developing services using the Globus Toolkit (GT) versions 4.0 and 4.2. As such, the document assumes that the reader is familiar with the basics of SOAP, WSRF and the programming model used by Java WS Core in GT 4.0 and 4.2. Similarly, this does not describe the entirety of the Crux Toolkit, but outlines the initial features to support use cases as requested by existing users of GT.


The following actors have been identified as being the targets of our use cases and stories. As such, the design goals were driven by the needs of:

  • Service Developer: Designs and develops services that interact in a Grid Environment.
  • Service Deployer: Deploys the service in a hosting environment and configured service properties, including security properties.
  • Resource Administrator: Owns and manages resources supporting the hosting environment on which services are deployed
  • Human Service User: Human clients that use the functionality provided by the service.
  • Software-Based Service User: Computer Software clients that use the functionality provided by the service
  • Infrastructure Developer: Designs and develops modules that can be used in conjunction with the service hosting environment to provide additional features to be used by Service Developers.

User Stories

A set of User Stories have been collected that reflect specific situations relevant to this proposal and drive the initial set of requirements. This list is not exhaustive and the proposed solution may address additional use cases. The initial set of user stories is listed below:

As a client, download metadata about a service

A Service User (Human or Software-Based) uses a well-known property name to retrieve metadata about a service.  That metadata should include, but not be limited to: service host/author, syntax and format of the operational signatures and the data used by the service's operations.

As a service developer, have my service indexed by a central index server

A service developer deploys a service into a service hosting environment. One or more service discovery agents is made aware of the newly deployed service and the service's metadata is appended to the discovery service's index.

As a client, use a discovery service to locate services of interest.

The client connects to a discovery service. The client submits a query for a type of resource or service they want. The discovery service scans its repository and returns a list of the services that match the query request.


  • It is desirable for this discovery service to support indexing of legacy services utilizing WSRF interfaces.

As a client, receive notifications about events associated with my resources

A client creates or discovers a resource that they want to monitor. That client may subscribe to event notifications from the resource. When the resource changes, a notification is sent to each of the subscribed clients.

As a client, specify the lifetime of a created resource

A client creates a resource via a service. If the resource has a specific lifetime (as determined in the service configuration), the resource is scheduled for destruction based on the lifetime. When either the resource lifetime expires or the resource is destroyed explicitly by some service operation, the service destroys and cleans up the resource. The client may also be authorized to extend the lifetime of the resource as it is actively using it.

As a client, access stateful resources even in the event of a server restart.

A client creates a resource via a service. The client interacts with the service for a while. The client can access that resource until the resource is explicitly destroyed, either by the client or by the service via a lifetime expiration.

  • Implicit is the notion that the resource should survive changes to its hosting service, such as a container restart, etc.

As a service developer, port my existing WSRF services to the new API.

A Service Developer wishes to port his existing WSRF legacy service the new API provided by GT The developer can migrate the service without reconstructing its basic business logic.

As a resource administrator, deploy a service artifact in my existing infrastructure.

A Resource Administrator is given an artifact of a service from a Service Developer. The Resource Administrator can deploy that artifact in the existing infrastructure.

As a resource administrator, monitor services running in my infrastructure.

A Resource Administrator is responsible for monitoring and maintaining the set of services running in the existing infrastructure. The Resource Administrator can use existing tools, to monitor service such as status, load and log messages.

As a client, secure access to service, as determined by service metadata

The client contacts a service endpoint anonymously (without specifying identity). The client invokes a well-known operation to retrieve the security metadata associated with the service. The metadata specifies the authentication policies for the service and any other resources associated with the service. The client parses security metadata for a service and uses API to set up security on the outgoing call. If the security requirements specified in the security metadata cannot be met, the client fails. Otherwise, the client invokes the desired operation on the service.

  • Desirable for the security metadata to be WS Security Policy and application level method replaced by some standard protocol to access the metadata.
  • The notion of "authorization" metadata has also been discussed; might be a separate user story. 

As a client, use X.509 Certificate/Proxy Certificate as credentials for authentication

The client uses the X.509 Certificate/X.509 Proxy Certificate to establish a secure transport connection with the service. The service validates the certificate chain and if found invalid, an exception is raised. The client validates the server's certificate chain and if found invalid, an exception is raised. The client sends the request to the service over the established secure channel.

As a service developer, expose service security (authentication) policy as service metadata

A Service Developer exposes the security policy requirement as part of a service's metadata. This policy includes the authentication method and parameters required for the authentication mechanism. A single policy might be configured across all instances of state associated with the service or individual policy might be configured for each state.

As a service deployer, configure service access policy

The Service Deployer configures the security policy such that access to the service is allowed only if an incoming service request is compliant with the policy. The policy includes authentication and authorization policy.

As a service developer, enforce service security policy for all client access

The client invokes an operation on a service. The service validates if service invocation is compliant with the authentication policy configured with the service. If not compliant, an error is returned to the client, otherwise, the service ascertains the identity of the client making the request. The service enforces the authorization policy associated with the service. If the client is authorized, the service invocation continues, otherwise the client is notified of the security exception.

End to end security

A web application pushes a message onto an Enterprise Service Bus (ESB). The bus invokes operations on grid service(s). The grid service verifies the request is from the web application. If it is, the operation is performed otherwise an error is returned.

  • Today this is solved by inbound/outbound service sitting on the bus to which web application delegates credential

These user stories should encompass most of the behavior supported in GT 4.0 and 4.2.

Globus Toolkit 4.x WS Framework

This section is a brief overview of the Globus Toolkit 4.x. This overview is included to set up a review of the opportunities for improvement and to compare how Crux Toolkit improves on the current solution.

GT Java WS Core

GT Java WS Core (known as Java WS Core) provides tools for service deployment, stateful services, state management and state persistence, as well as providing an implementation of the WS-ResourceFramework (WSRF) specification ( The modules heavily rely on Apache Axis 1.x, as well as Apache Directory, Apache Digester, Apache Addressing, Apache Commons and Doug Lea's Concurrency Library.

Java WS Core can be viewed as a set of layers:

At the lowest level, Java WS Core uses GT security ( for authentication and authorization. The next layer is is the Application Engine. Java WS Core provides it's own implementation of an Application Engine called the GT Service Container, and provides support for deploying the toolkit into Tomcat or JBoss. The GT Service Container was originally designed to be used mainly for testing purposes, but has become one of the most common deployment techniques.  The GT Security Handlers, GT Addressing Handlers and GT Service Dispatcher are all implemented as SOAP handlers. The GT Security Handlers support authentication, authorization and security policy enforcement. The GT Addressing Handlers are responsible for processing the WS Addressing headers, which are not supported by Axis 1.x, but are required for WSRF.  The final handler, the GT Service Dispatcher, is an extension of the dispatcher in Axis 1.x and is responsible for directing requests to the appropriate service object or operation providers, which are utility objects that provide many of the WSRF operations.  The next layer up represents service logic code.  These objects consist of the user's service objects as well as the WSRF operation providers, previously described. At the top level are a collection of useful tools and services designed to help build service objects such as an implementation of JSR 237 ( for a WorkManager, resource management components amongst others.  These can, optionally, be used by the service developer.


The limitations of the Java WS Core largely stem from the age of the code, tight coupling of third party dependencies and the SOAP requirement.

Several third party library dependencies, including Apache Directory and Apache Addressing are no longer supported by the Apache community and must be maintained by the GT team. Some other dependencies are customized and maintained by the GT team. Better tooling to support the features provided by these third party library are available, but the tight coupling of Java WS Core with specific versions of the existing dependencies precludes switching to newer tools.

In addition, the Java WS Core maintains it's own Application Server, duplicating the functionality of mature, stable Application Servers available. The Java WS Core Application Server neither performs as well as other stable Application Servers like Apache Tomcat and Jetty, nor is it as configurable.

Java WS Core suffers from some major performance limitations with XML processing due to its dependency on Axis 1.x. Axis 1.x also uses an idiosyncratic deployment model which does not provide support for functions such as hot deployment. Nearly every piece of Java WS Core has a dependency on Axis 1.x, since not only is Axis 1.x used as the SOAP engine for Java WS Core, but it is also used for the XML object binding. As such, migrating the existing code to a newer SOAP engine, with better performance, is not feasible.

This component also lacks support for non-SOAP interfaces, such as POX/http and imposes use of WS Addressing specification on all services developed using the toolkit. Lack of sufficient administration and monitoring tools renders managing production deployments of Java WS Core difficult.

Over all, the age, lack of modularity and high level of coupling to obsolete third party libraries creates a significant set of limitations to productive use of the Java WS Core.

GT Java Security

This section provides a brief overview of the features provided by the various GT Java Security WS components.

The GT Java WS Security provides authentication, authorization and security policy enforcement for web services written on Java WS Core. The module leverages a third party libraries, CoG JGlobus and CoG JGlobus Feature Extensions (FX) for various security features like the GSI protocol, Apache Tomcat filters for GSI and Gridmap file implementation.

Security pieces of CoG JGlobus module provides the GSS API implementation of GSI and Gridmap file management. The implementation supports both the GSI mode, which allows delegation of credential as part of the protocol and SSL compatible mode. The module also contains Java command line clients for user certificate management, proxy certificate management and MyProxy server interactions. The feature extension module, CoG JGlobus FX, provides valves and connectors for use of the GSI protocol with Apache Tomcat 4.x, 5.0.x and 5.5.x. In addition, the module extends the Apache Commons HTTP Client library to provide connection pooling support for HTPS connections that use the CoG JGlobus implementation of the GSI protocol. This connection pooling support allows for reuse of connections to a given endpoint, created with the same security properties.

The WS Security provides support for authentication and message protection using Secure Transport, WS Secure Message and WS Secure Conversation. By default, mutual authentication is done and the security mechanism used by the client for service access is mimicked in the response. Authorization of service access is supported by leveraging the GT Java Authorization module, a WS agnostic authorization engine, that allows pluggable authorization policy and combining algorithms to be enforced. The WS Security also allows for enforcement of security policy, such as, the authentication mechanism, authorization policy and message protection, for access to the service. The security policy can be specified and enforced at the hosting environment, service or resource (instance of state) level, using configuration defined by schema files shipped with the toolkit.

Protocol agnostic delegation is provided in the toolkit using the Delegation Service. The service is built on the Java WS Core component and provides a remote interface to delegate and manage the delegated credentials. The service allows for co-hosted services to access the delegated credential for use.


The Java WS Security implementation in GT is tightly coupled with Java WS Core and has direct dependency on Axis 1.x. The poor performance of XML handling in Axis 1.x is also reflected in the GT security code due to the direct dependency on the software. Some part of the security support assumes the use of WS Addressing and is tied to the SOAP protocol. This renders the module unusable for front-ends like REST.

Additionally, the security code relies on some third party libraries that are not supported any more and the maintenance burden falls on the Globus team. the security code also relies on customization of some old versions of libraries like WSS4J. The various security features are tightly coupled, forcing service developers to deal with all the security modules rather than choose a required subset. Even independent modules like CoG JGlobus and Java Authorization require libraries from Java WS Core to build and have poor dependency management, thus rendering its use outside of core and, in general GT, an onerous task.

The Delegation Service, while providing a protocol agnostic way of sharing delegated credentials, requires that the consumer of the credential exist in the same hosting environment. This requirement greatly limits the scope ad usage of the service.

Globus Crux Toolkit

Since GT 4.0 was written, the field of Web Service Development Kits (WSDK) has expanded dramatically. Major open source WSDKs include: Axis 2, CXF, Metro, and Spring WS. However, none of these toolkits provide all of the tools required to support the user stories listed above.  To this end, we propose a new version of Globus WS Core and Security which we call Globus Crux, that can be built/deployed on existing WSDKs to augment the supported feature set.

The primary goal of the Crux Toolkit is to support and provide tooling and common services for service authors who want to deploy their services in a grid environment. A secondary goal of the Crux Toolkit is to provide tools that are not susceptible to vendor lock-in, which was a significant problem with Java WS Core 4.x. To facilitate this, the Crux Toolkit will focus on supporting specifications and standards defined as Java Specification Requests (JSR) as part of the Java Community Process (JCP). While not all of the JSRs which result from the JCR will have significant adoption, Crux can exploit the popular JSRs to provide support for features which will, likely, be supported by a number of vendors and, as such, avoid the vendor lock-in problem.

However, the performance benefit of directly leveraging vendor specific API far exceeds the advantages of maintaining agnostic code.  While the toolkit will be largely based on existing standards, the reference implementation will be based on, and as a result, linked to the Apache CXF WSDK. Apache CXF is a full featured WSDK with support for WS-Addressing, WS-Security, WS-ReliableMessaging amongst other specifications. CXF is a toolkit with good performance as a result of it's dependency on modern XML processing tools. Simply put, because of its features, performance and support from Progress Software, CXF is an ideal toolkit on which to base Crux. For details on the evaluation of the available WSDKs, please see WSDK Evaluation

In this section we outline the planned initial features for Crux Toolkit.

Stateful Services

User feedback indicates that stateful and resources exposed as servicesare a major requirement for the Web services. For some Web services technologies, such as JAX-RS (for support of RESTful services) state management is, to some degree, inherent. RESTful services are designed to  provide a clear mechanism for identifying a resource. Most distributed computing technologies, especially SOAP based services, do not provide a default mechanism to associate state identity with a message. (N.B. Some implementations of the Java Message Queue (JMS) specification support message groups via the JMSXGroupID header which provide a form of state management). In order to support all of these technologies, Crux aims to provide a mechanism for providing state management to a wider collection of service technologies. This support will allow service developers to access the stateful id information in the scenario where stateful services are not inherent to the messaging technology and also provide a simple method of resolving the state based on the stateful Id for both "stateful" and "non-stateful" services. This should shield the service developer from the burden of handling the state associated with a service request.

Persistent Resources

Crux is designed to provide support for resources which persist indefinitely, even across server restarts. There are, at present, a wide variety of technologies available for persisting objects. These include: Java Persistence API (JPA), Java Data Objects (JDO), Hibernate, Cayenne, Castor and a wide variety of others. All of these technologies provide strong support for mapping Java objects to some persistent data store (often a relational database system or RDBMS).  Crux will provide a plug-able persistence framework which allows the service developer to one of the plug-ins provided by Crux to enable persistent resources or create a custom implementation which will work within the Crux framework.

Integrated Messaging

Many of the services that will be based on Crux will function asynchronously. The presents a need to support some form of event messaging to notify clients and others about the status of requests. There seem to be a plethora of messaging standards at the moment including JMS, AMQP, XMPP, Cometd, WS-Notification, WS-Eventing as well as many others. Java WS Core 4.x provided extensive support for WS-Notification, but no support for these other technologies. In addition, the API for interacting with the messaging system was non standard.   Crux will provide a messaging framework as well, but based on JMS. This means that service developers will be able to interact with a standard API which is vendor independent. Modern JMS engines such as ActiveMQ have an architecture based on plugable transports. As a result, new messaging specifications such as WS-Notification could be plugged into an existing engine with no impact on the service developer. The Crux framework will provide integrated support for notifications based on changing state of resources as well as a seamless way for services to interact with messaging technologies.

Monitoring and Management

Grid deployments require significant management to maintain due to their distributed character. Historically, monitoring and management of such systems has been a complicated and (often) frustrating experience. Crux will provide a set of monitoring and management tools designed to simplify the work of the service deployer. The monitoring services will be based on the Java Management Extensions (JMX) technologies , thereby supporting a well supported Java standard. Crux will provide a web based management console that will allow a Resource Adminstrator to easily monitor the status of services as well as upgrade, deploy and undeploy services. Because the monitoring services will be based on JMX, administrators and developers will be able to use existing monitoring consoles or develop their own.

Deployment Tools

Crux will provide a collection of best practices which will help guide the service deployer through the maze of enterprise service deployment. While Crux will not provide a deployment container itself (the Enterprise Application Server industry has a wide variety of containers designed to support most configurations), Crux will provide documentation on how to integrate with leading deployment technologies. In addition, Crux will provide plug-ins and tools to integrate directly with some application server technologies. Doing so will not restrict users, but will provide extra capabilities for users that wish to take advantage of these deployment technologies.

Discovery Tools

In order for many grid services to be useful, clients need a way to discover what resources and services are available and metadata about those services. While Crux will not be implementing a Discovery Service, another Globus group is building a new, high performance service. Crux will provide a data model and API to interact with this service. More details about this service are forthcoming.


Based on the user stories, securing of Web services using TLS and message level security, with support for delegation and single sign-on credentials, are key requirements. While existing toolkits provide basic support for both forms of security, the use of X.509 Proxy Certificate for both automated processing and delegation are not supported. Crux security will provide tools to manage X.509 Proxy Certificates and validate them, in addition to modules for using Proxy Certificates for TLS. This will allow for single sign-on and use of current credentials with the new toolkit.

User stories also indicate a requirement for sophisticated authorization framework that allows pluggable authorization policy to be evaluated and enforced. The GT Java Authorization Framework is a module that provides users with pluggable authorization policy and combining algorithm, enabling leveraging of custom policies and legacy policy engines. The framework ships with built-in providers for construction of authorization decisions based on delegation of rights from the resource owner to the requester and there are numerous policy evaluation pieces developed and maintained by the community using this framework. Crux Toolkit will will include the above framework and any integration layers required for leveraging it for authorization of service invocation.

Delegation Services provide a store for a client's delegated credential, which can be shared with a resource that needs to act on behalf of the client. While Crux itself will not provide a Delegation Service, the Globus team is working on a Delegation Service implementation and the Crux Security pieces will provide API and tools for service developers to be able to interact with it. The tools and API will include the ability to delegate credentials, manage delegated credentials and access delegated credentials.

Backwards Compatibility with GT 4.x

Given the large number of existing Globus deployments, though, a goal of Globus is to maintain some backwards compatibility via wire protocols without introducing significant maintenance requirements. Globus will also provide well documented migration paths for users taking advantage of older services.

Given that most of the WSRF simply describes a set of SOAP operations, supporting a subset of these operations in a non-intrusive way is necessary. Like many specifications, our users only use a portion of the operations supported in WSRF. By taking advantage of the ability of many modern SOAP engines to communicate over multiple transports, Crux will support a declared subset of WSRF, without requiring a coding effort from the service developer.

To this end, Globus will provide an implementation for a subset of the WS-RF specifications, which will cover the majority of use cases which existing users have reported. Specifically, Globus will continue to provide support for:

  • From WS-ResourceLifetime
    • Immediate Destruction
    • Scheduled Destruction
  • WS-BaseFaults
  • From WS-ResourceProperties
    • GetResourceProperty
    • GetMultipleResourceProperties
    • QueryResourceProperties
    • Subscription
  • Subset of WS Notifications (BaseNotifications)

GT 4.0.x Migration

The Globus Toolkit 4.0.x was implemented on top of draft versions of the specifications specified above. We aim to provide support for clients  using these versions of the specifications. Services will need to update their implementations to map to the newer API, but clients should not be affected. The older specifications will be deprecated over time as appropriate.

GT 4.2.x Migration

The Globus Toolkit 4.2.x was implemented on top of the final specifications above. Clients using these specifications should not notice any difference in functionality. Services will need to migrate to the newer API's provided by Globus, but they will remain wire compliant with older clients

Crux Reference Implementation

As stated above, the initial implementation of Crux will be based on Apache CXF CXF will provide the backbone for SOAP services. Crux will provide an implementation of the subset of WSRF list above that runs on CXF via JAX-WS. As was the case in GT 4.0 and 4.2, the WSRF support will be provided through Operation Providers. These will be JAX-WS based implementations of the WSRF specifications. Crux will then use CXF to route the messages from the external port to the internal implementation of the provider. This should lead to a smooth transition for developers who are currently taking advantage of Operation Providers in GT.

Stateful services and ResourceProperties will be implemented using Java Annotations in Crux.  One consequence of this is that Crux will require Java 5. These annotations will allow developers to declare stateful resources as well as fields and method which provide ResourceProperties.  This approach requires significantly less code than what is currently required with the Interface approach in GT. In addition, the stateful annotations decouple the service from the method by which the state is maintained. The service simply operates on a piece of state, regardless of how the state is maintained. Crux will, initially, provide implementations of the persistence plug-in based on JPA and JDO.

As was described above, events and notifications will be implemented on top of JMS interfaces. This will give the notification mechanism a more standardized API and facilitate different methods of notification.  ActiveMQ will be the base implementation for the JMS support. ActiveMQ provides support for a wide array of transports and is a mature and robust JMS implementation.  One side effect of the use of JMS to support notifications is that creating non-resource based notification sources will be significantly simpler than it was under GT.

As mentioned before Crux Security will support PKI framework with X.509 Certificates and delegation of credentials. The Crux Security pieces will be ship in multiple modules. A Crux Security Module that contains the basic security pieces and is agnostic of WS Toolkits will be the core security module. This module will contain core security pieces and will be developed with portability as an important factor. A Crux Security Plug-in module targeting Apache CXF will be developed, that builds on the Crux Security Module to integrate the security pieces to the specific WS Toolkit in the most performance efficient way. For example, the plug-in module will tie the use of X.509 Proxy Certificate with WS Secure Message on Apache CXF, which implies it will integrate with WSS4J also. Another example will be a plug-in module that will provide valves and filters required for using X.509 Proxy Certificate with Apache Tomcat. A preliminary module based on Apache CXF, and hence WSS4J, will be shipped a Crux Plug-in. Support for integration with other WSDKs will be driven by user requirements and resource availability, but security modules will be developed with portability as an important factor.

Crux Security will provide an integration layer for services written on Apache CXF to leverage the GT Java authorization framework, thus allowing the community to use the existing implementations. While the module itself will continue to be agnostic of WS Toolkits, an integration layer specific to the target toolkits will be provided.

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Jul 24, 2009

    oster says:

    Some comments: The Actor of "Resource Administrator" is a bit misleading in th...

    Some comments:

    • The Actor of "Resource Administrator" is a bit misleading in the context of a WSRF framework; perhaps rename "Infrastructure Administrator" or something like that (Hardware? Platform?).
    • The defined Actors are not used in all the user stories (e.g.. many say "As a client, ...."); either use them consistently, or define a "Client" actor as more general Human/Software-based Service User
    • These two statments are somewhat contradictory (I think the first one needs to be clarified):
      • Crux will support a declared subset of WSRF, without requiring a coding effort from the service developer.
      • Services will need to update their implementations to map to the newer API, but clients should not be affected.
    • The first questions will be about "timelines"; we should at least acknowledge this somehow, if even to say how we will be coming up with timelines we don't yet know.
    • I think the introduction could be beefed up a bit in terms of justification.  It reads as though aging technologies and performance are the primary drivers for change, and therefore the only benefits of that migration.  We should at least comment on all the added benefits this will enable.