RSA Archer Upgrade - Fixing a GemStone.GemFire.Cache.DLL Error

If you've been working with RSA Archer long enough, you know that upgrading or installing patches to your RSA Archer instance can be a pain. Even when you think you have everything perfectly planned, the upgrade or patch installation can still have problems. Lately I've seen and heard of folks having problems getting their instance to launch after an attempted upgrade. When they ask for help, they're instructed to reinstall the application as a fix; this can take a long time and sometimes you can lose data as a result.

My advice? Do not acquiesce to a RSA Archer reinstall until you've tried properly troubleshooting the application. There's a lot you can try to fix your application before succumbing to a reinstall, and for busy Archer developers and administrators, properly fixing the application versus reinstalling can save a lot of time.

Here's how to fix what appears to be a generic error message, but turns out to be related to a very specific DLL called the GemStone.GemFire.Cache.DLL. This fix would apply to almost any DLL.

After an attempted upgrade, Archer returns the following error in a web browser:

Server Error in '/' Application.
Runtime Error Description: An exception occurred while processing your request. Additionally, another exception occurred while executing the custom error page for the first exception. The request has been terminated.

This error doesn't provide much information. Something you should know: By default, Archer doesn't produce verbose error messages and therefore is not "troubleshooting friendly." You need to enable detailed error messages so you can start tracing the source of error.

To enable detailed error messages logging you need to turn off custom error messages. To do this, find your web.config file, usually located in a directory that looks similar to this: C:\inetpub\wwwroot\RSAarcher\web.config. Find where the files reads:

<customErrors mode="On" defaultRedirect="error.aspx" />

Edit this line to read <customErrors mode="Off" defaultRedirect="error.aspx" /> and save the web.config file. You can try refreshing your browser, but you may also need to restart the application pool in your ASP.NET installation in order to see a verbose error message. As a side note, once you are done with troubleshooting, you should restore the customErrors mode="Off" back to "On" or "RemoteOnly" for security purposes. Once you've made the web.config configuration change, your browser might read something like this:

Server Error in '/' Application.
Could not load file or assembly 'GemStone.GemFire.Cache.DLL' or one of its dependencies. The specified module could not be found.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.IO.FileNotFoundException: Could not load file or assembly 'GemStone.GemFire.Cache.DLL' or one of its dependencies. The specified module could not be found.

Source Error:
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

In this specific case, there's an issue with the GemStone.GemFire.Cache.DLL file. Going back to the previously mentioned web.config file and searching for where the GemStone.GemFire.Cache.DLL is referenced, we can see where the DLL file is actually located.

Extract from web.config file:

<runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="GemStone.GemFire.Cache" publicKeyToken="<snip>" culture="neutral" />
        <codeBase version="7.0.1.0" href="./gemfirebin/GemStone.GemFire.Cache.dll" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>

From here, we can tell the GemStone.GemFire.Cache.dll is stored in the gemfirebin folder. Remember, you can search windows server for specific files or folders.

Upon further examination, in this case the Archer GemStone.GemFire.Cache.DLL file was missing after the upgrade, and therefore the file couldn't be loaded and the application wouldn't launch. Once the GemStone.GemFire.Cache.dll was restored from a backup, the web server restarted and the application launched with no problems. Now, the tricky part can be figuring out which version of the DLL you need, and whether the existing DLL should work or if the DLL was updated in the new version of Archer. Calling RSA Archer support or your trusted Archer integration partner to determine which DLL you need.

This is a common application troubleshooting process. The process of reading logs, tracing issues back to specific files/actions, making an attempted fix per the error message, and restarting the server should be repeated until all error messages are resolved.