General Issues with Tomcat Deployments

The Problem

Attempting to execute Tomcat from the command line results in it failing to start correctly and a report similar to the following on the console:

C:\web-servers\SERVICEManager\tomcat\bin>tomcat8

[2017-04-18 13:30:57] [warn]  [ 5952] The system cannot find the Registry key for service 'tomcat8'
[2017-04-18 13:30:57] [error] [ 5952] Load configuration failed
[2017-04-18 13:30:57] [error] [ 5952] The system cannot find the file specified.
[2017-04-18 13:30:57] [error] [ 5952] Commons Daemon procrun failed with exit value: 2 (Failed to load configuration)
[2017-04-18 13:30:57] [error] [ 5952] The system cannot find the file specified.

 The Reason

The Tomcat application server can be run either as a service, or directly from the command line.  To run in either mode Tomcat needs to be INSTALLED as a service because, regardless of start up mode, Tomcat attempts to retrieve its start up settings from the registry.  Part of the service installation routine populates the registry. 

The Resolution

Install ServiceManager as a service using the supplied install_tomcat_service.bat script file

The Problem

Attempting to execute Tomcat/SERVICEManager after installation using Tomcat's service.bat installation script results in ServiceManager failing to start correctly.  Connection via a browser results in the browsers failing to display the logon page and display the following stack trace information:

root cause 
java.lang.IllegalStateException: ServiceManagerProperties.instantiate
must be called before any use of this class
com.servicepower.servicemanager.ServiceManagerProperties.getInstance(ServiceManagerProperties.java:60)
org.apache.jsp.jsp.ServerExceptionNotification_jsp._jspService(ServerExceptionNotification_jsp.java:149)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)

Review of the stdout/console log for Tomcat 

SERVICEManager: setUpPropertiesFile - started
SERVICEManager: setUpPropertiesFile - using plug-in value:
\sm_properties\service_manager.properties
SERVICEManager: ** ERROR ** setUpPropertiesFile could not access
the file file: \sm_properties\service_manager.properties
java.lang.IllegalStateException: Could not access
service_manager.properties file
        at
com.servicepower.servicemanager.ApplicationSetupPlugIn.setUpPropertiesFile(ApplicationSetupPlugIn.java:131)
        at
com.servicepower.servicemanager.ApplicationSetupPlugIn.init(ApplicationSetupPlugIn.java:365)
...
SERVICEManager: init - ended: unsuccessfully

The Reason 

 Each ServiceManager release comes with a script file: install_tomcat_service.bat responsible for registering the installed Tomcat instance as a service.  ServiceManager does not perform the service registration itself, but defers to the Tomcat-supplied service.bat file.  This file should not be called directly as ServiceManager requires a number of specific environment settings to be defined prior to its invocation to service.bat.  By calling service.bat directly Tomcat will be registered as a service but will not have the correct ServiceManager start up parameters.

The Resolution

 Uninstall the service using the Tomcat supplied script file:

cd [SERVICEManager-root]\tomcat\bin
service remove

And install the service using the ServiceManager supplied script file:

cd [SERVICEManager-root]
install_tomcat_service.bat

The Problem

This occurs when the ServiceManager service is configured to use an incompatible version of Java. Starting the Tomcat service results in the following error being displayed:

Looking in the Tomcat logs (SERVICEManager\tomcat\logs), the commons-daemon log will show the following if the Java version is incompatible:

[error] [ 4512] %1 is not a valid Win32 application.
[error] [ 4512] Failed creating java C:\Java\32bit\jre1.6.0_45\bin\client\jvm.dll
[error] [ 4512] %1 is not a valid Win32 application.

And it will show the following if the path to Java is not valid:

[error] [ 2456] Failed creating java C:\wibble\java\jre\bin\server\jvm.dll
[error] [ 2456] The system cannot find the path specified.

The Reason

ServiceManager, and Tomcat, have been implemented against Java 8. The install_tomcat_service.bat script configures the service to run against the Java Runtime Environment (JRE) that comes bundled with ServiceManager, so this problem will only arise if someone has attempted to change the service to use a different Java version or the version it is looking for has been accidentally moved or deleted.

The Resolution

Either:

Uninstall and reinstall the Tomcat service to configure it to use the JRE bundled with ServiceManager.

Or:

Update the Tomcat service's configuration to correct the path to Java using the tomcat8w.exe tool which is found under SERVICEManager\tomcat\bin. By default this updates the service named Tomcat8; to provide a different name, use the optional //ES// command line argument, eg:

tomcat8w.exe //ES//newTomcat8

On the Java tab, set the Java Virtual Machine path to:

<path to SM>\SERVICEManager\java\jre\bin\server\jvm.dll

The Problem

The machine has an existing ServiceManager installation set up to use Tomcat as a service, the new ServiceManager fails to install as a service, reporting:  

C:\SERVICEManager>install_tomcat_service.bat install

** install_tomcat_service.bat - Tomcat/SERVICEManager service installation routine **
No service name supplied - assuming default
...
Installing the service 'Tomcat8' ...
Using CATALINA_HOME:   	C:\SERVICEManager\tomcat
Using JAVA_HOME:    	C:\SERVICEManager\java
Using JVM:           	C:\SERVICEManager\java\jre\bin\server\jvm.dll
Failed installing 'Tomcat8' service

The Reason

Installing Tomcat as a service results in a number of entries being written to the registry.  If no name is specified during the service installation the default name of Tomcat8 is used.

Attempting to install a second Tomcat instance without specifying a service name, or specifying the same service name, results in the preceding failure.  

The Resolution

Either remove the originally installed service, or provide this new service with a name:

C:\SERVICEManager>install_tomcat_service.bat install anotherName

The Problem

Two (or more) installations of ServiceManager installed on the machine and for some reason the second ServiceManager/Tomcat instance is using the former's environment settings.  This typically manifests itself in both ServiceManager instances connecting to the same database and ServiceOptimizer instances.

The Reason

As described earlier execution of Tomcat relies of reading a set of registry settings.  When executing without specifying a service installation name the Tomcat instance will use the settings specified for the default installation (if applicable).  If a service instance name is specified Tomcat will execute using those settings, regardless of whether the service name was indeed correct for this instance.

The Resolution

Check that both Tomcat instances are being launched with the correct (in terms of how the service was registered) service name as this is used to determine which registry settings are retrieved and utilised.

The Problem

To deploy a new version of ServiceManager its web-application-archive file (.WAR) needs to be dropped into Tomcat's webapps directory, whilst Tomcat is not executing.  

If Tomcat is restarted before the old expanded archive has been deleted then attempts by Tomcat to serve the ServiceManager logon page will result in a HTTP 404 error:

The Reason

The problem has occurred because, on start up, Tomcat has deduced that there is a newer version of ServiceManager and has attempted to redeploy it.  However the redeployment has failed because the original expanded archive has not been successfully deleted by Tomcat.  Once in this state, Tomcat believes that it has redeployed ServiceManager and it is available for servicing.  However because the expanded web application no longer contains a valid application Tomcat fails to serve it correctly.

The Resolution

It is not possible to delete the expanded application directory whilst Tomcat is executing, so Tomcat must be terminated before it is possible to manually delete the webapps\SERVICEManager directory.  Restarting Tomcat will result in a successful deployment of the new version.

  The Problem

The same as the preceding issue except that in this case Tomcat was not terminated and tried to redeploy automatically.  After, the failed, redeployment Tomcat fails to serve pages as described previously.

The console log for Tomcat will display information similar to:

INFO:Undeploying context [/SERVICEManager]
SERVICEManager:destroy - started
FileDeleter thread ended
SERVICEManager:FileDeleterThread.isAlive = false
SERVICEManager:destroy - ended
13-Dec-2006 10:26:48 org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive SERVICEManager.war

The Reason

As with the preceding problem.

The Resolution

As with the preceding problem.

The Problem

After successful deployment of a new ServiceManager instance Tomcat is restarted.  However some of the pages being served (for example the logon page) are the same as the previous version.  Similarly, a page within the new deployment still exhibits a fault that had been described as fixed.

The Reason

There are two probable causes of this problem.

1. The browser cache has not been cleared and so has not loaded the latest version
2. Tomcat's JSP cache was not deleted prior to Tomcat's restart after deployment

The Resolution

1. Clear the browser cache (also known as browsing history) then reload the page.
   If this does not work, then
2. Stop Tomcat, delete the cache directory ( SERVICEManager\tomcat\work\ Catalina\localhost\SERVICEManager ) for the appropriate web application then restart Tomcat.

The Problem

After restarting Tomcat, ServiceManager fails to serve some pages correctly and the browser displays a stack trace for a java.lang.NoSuchMethodError, or possibly a java.lang.ClassNotFoundException

The Reason

JSP technology allows its pages to include not just HTML but also Java objects.  If as intimated in the previous problem Tomcat's cache of JSPs has become out of date with the underlying ServiceManager Java code, and some of the Java code the JSPs depend upon has had their contracts modified (i.e. interface changes) then the runtime binding of the JSP those objects will fail.

The Resolution

Manually delete the appropriate JSP cache directory

The Problem

Two versions of Tomcat/SERVICEManager have been deployed and started, however the second instance fails to start correctly and terminates with the following displayed in the console log:

SEVERE: Error initializing endpoint
java.net.BindException: Address already in use: JVM_Bind:9090 at org.apache.tomcat.util.net.PoolTcpEndpoint.initEndpoint(PoolTcpEndpoint.java:297)

...
14-Dec-2006 10:32:54 org.apache.catalina.startup.Catalina load
SEVERE: Catalina.start
LifecycleException: Protocol handler initialization failed: java.net.BindException: Address
already in use: JVM_Bind:9090 at org.apache.catalina.connector.Connector.initialize(Connector.java:1018)
...

The Reason

By default Tomcat instances expect to use port 8080, so without any modifications to its configuration the second Tomcat instance will attempt, and fail, to bind to port 8080.

The Resolution

There are two levels of configuration for each Tomcat/SERVICEManager instance: the configuration required to define where the ServiceManager properties and configuration files reside, and configuration of the Tomcat instance.

Tomcat configuration is centred upon modifications to the use tomcat\conf\server.xml file.  This contains the port definitions to be used by the particular instance.  As a minimum two port definitions need changing to avoid contention: those defined within the <Server> and <Service> elements:

<Server port="9005" shutdown="SHUTDOWN">
...
	<Service name="Catalina">
    	<Connector port="9090" maxHttpHeaderSize="8192" .../>
    	...
  	</Service>
...
</Server>