Upgrading a Cluster with a Separated Database and Data Hub Module

Prerequisites:

This procedure describes how to upgrade Spectrum™ Technology Platform when you have separated the configuration database from the server and you have the Data Hub Module installed. The upgrade process consists of these steps:

  1. Stop the server cluster.
  2. Stop the configuration database cluster.
  3. Upgrade the last node that was stopped in the configuration database cluster.
  4. Upgrade the other nodes in the configuration database cluster.
  5. Upgrade the server acting as the Data Hub Module master in the server cluster.
  6. Upgrade the non-master servers in the server cluster.

To upgrade a cluster with a separated configuration database and the Data Hub Module, follow this procedure:

  1. If you do not already have a backup copy of your models, create a backup manually. To create a backup manually, open the Relationship Analysis Client and click Manage. Select the model you want to back up then click Backup.
    In addition to backing up your models, back up these two property files:
    • server\modules\hub\hub.properties
    • server\modules\db\neo4j.properties
  2. Identify which node is the serving as the master server for the Data Hub Module.
    1. Open a web browser and go to:

      http://LoadBalancer:8080/jmx-console/HttpAdaptor/list

    2. Scroll down to the neo4j.org domain.

      You will see a set of objects for each model. The Role attribute in the HighAvailability object indicates whether a server is the master for a model.

    3. If you have more than one model and each model has a different master, you need to make one server the master of all the models. To do this, restart the cluster then open the Relationship Analysis Client using the hostname or IP address of one of the nodes rather than the load balancer. In the Relationship Analysis Client, open each model by running a query on each model. This will make the server you are connected to the master for each model.
  3. Stop all the non-master nodes in the server cluster then stop the master server last. Stop nodes one at a time rather than all at once.
    Important: Make sure that Spectrum™ Technology Platform stops without errors. If a server does not stop properly, Data Hub Module models on the server will not open successfully after upgrading. To ensure that Spectrum™ Technology Platform stopped cleanly, examine the SpectrumDirectory\server\app\repository\logs\wrapper.log file for errors during shutdown.
  4. Stop all the nodes in the configuration database cluster. Make a note of the last node that you stop in the configuration database cluster. You must start this node first after upgrading.
  5. On the last node that you shut down in the configuration database cluster, execute installdb.exe. The installer upgrades the configuration database.
    Warning: The first node that you upgrade must be the last node that was stopped. This is because on some operating systems the configuration database will start automatically at the end of the upgrade process. If the first node that starts is not the last node that was stopped, data such as job history and configuration settings may be lost. If you do not know which node was stopped last, look in each node's wrapper log for the time stamp of the shutdown message. You can find the wrapper log in: Spectrum Location\server\app\repository\logs\wrapper.log.
  6. After the upgrade process finishes, wait for the server to start, then stop the server.

    You can see when the server has started up by opening the log file <Spectrum Installation Location>\server\app\repository\logs\wrapper.log and looking for this message:

    INFO  [Server] Pitney Bowes Spectrum(TM) Technology Platform Database (Version version build) Started
    Important: Do not attempt to stop the server until after it has fully started for the first time. Stopping the server before it performs an initial startup can cause your installation to become unusable.
  7. Run the conversion utility.
    Note: You must run the conversion utility after upgrading the configuration database cluster and before upgrading the server cluster.
    • In order to run this utility, the server must have adequate disk space. The amount of free disk space needed is three times the size of the server/app/repository/store/databases folder. If necessary, add disk space before proceeding.
    • The conversion process may take an hour or more if you have a large configuration database.
    • If no conversion is necessary, the utility will display a message saying that there is nothing to do.
    1. Verify that all nodes in the configuration database cluster are shut down.
    2. On one of the nodes, back up this folder:
      server/app/repository/store/databases
    3. On the last node that was stopped in the cluster, right-click this file and select Run as administrator:
      SpectrumFolder\server\bin\convert.db.bat
  8. Configure clustering for the database cluster.
    1. Edit the file server/app/conf/spectrum-container.properties as follows:
      spectrum.cluster.name
      • If this is the first node in the cluster, set this to any name you want to give to the cluster. Be sure that the name is unique if you have multiple clusters.
      • If you are adding a node to an existing cluster, set this to the name of the cluster that you want this node to join.
      spectrum.cluster.enabled
      Set this to true.
      spectrum.cluster.address
      Set this to the network interface to which you want this node to bind. Specify 0.0.0.0 if you want the node to bind to all network interfaces.
      spectrum.cluster.seeds
      Set this to the IP address of the configuration database server you want to be used to set up new configuration database nodes added to the cluster. The configuration database from the seed server is copied to the node whenever it starts up so that it is synchronized with the cluster.
      Note: If this is the first node in the cluster you may specify seed nodes that are not yet running. If you know the IP addresses of the nodes you want to use as seed nodes, you may enter those now. Otherwise, you will need to return to this properties file once you have the IP addresses of the seed nodes and enter them.

      You can specify multiple comma-separated IP addresses. You should specify multiple seed nodes so that if one is down others can be used to allow the node to join the cluster. The node will attempt to access each seed node in the order listed. If none of the seed nodes can be reached, the node will not join the cluster.

    2. Save and close the spectrum-container.properties file.
    3. Start the last node that was stopped in the configuration database. To start the configuration database, right-click the Spectrum™ Technology Platform icon in the Windows system tray and select Start Spectrum™. Alternatively, you can use the Windows Services control panel to start the configuration database by starting the Spectrum Database service.
      Warning: The first node that you start must be the last node that was shut down. Starting up another node first may result in loss of data such as job history and configuration settings. If you do not know which node was shut down last, look in each node's wrapper log for the time stamp of the shutdown message. You can find the wrapper log in: Spectrum Location\server\app\repository\logs\wrapper.log.
    4. After the upgraded configuration database is fully started, repeat the previous steps to upgrade each of the remaining servers in the configuration database cluster and start each of them.
  9. Upgrade the Data Hub Module master server in the server cluster.
    1. If you are upgrading from Spectrum™ Technology Platform 10.0 or later, each model directory must contain a version.data file. Review all the model.ModelName subdirectories located under SpectrumDirectory/server/modules/hub/db to confirm that they contain a version.data file. If any model directory is missing this file, copy the corresponding version.data file from one of the non-master nodes.
    2. Run the Spectrum™ Technology Platform installer to upgrade the master server to the new version of Spectrum™ Technology Platform.
    3. After the upgrade process finishes, the Spectrum™ Technology Platform server starts automatically. Wait for the server to finish starting completely, then stop the server.
    4. Open the file SpectrumFolder\server\modules\hub\hub.properties in an editor and confirm that the property hub.neo4j.database.type is set to embedded:
      hub.neo4j.database.type=embedded
    5. Open the file SpectrumFolder\server\modules\hub\db\neo4j.properties in an editor and set the allow_store_upgrade property to true:
      allow_store_upgrade=true
    6. Start the Spectrum™ Technology Platform server.
    7. Open each model in the Relationship Analysis Client and run a query. Any query is sufficient.
    8. Stop the Spectrum™ Technology Platform server.
    9. Open the file SpectrumFolder\server\modules\hub\hub.properties in an editor and set the property hub.neo4j.database.type to ha:
      hub.neo4j.database.type=ha
    10. Compare any properties files that you backed up to the installed files and make any necessary changes. Do not overwrite new files with old files because new files may contain properties that old files do not.
  10. Upgrade the Data Hub Module non-master servers in the server cluster.
    Important: When installing each server select the Server only option in the installer and provide the host and port of one or more of the configuration database servers. You can find the port in the file InstallationLocation\server\app\conf\spectrum-container.properties. The port is specified in the spectrum.orientdb.binary.port property.
    1. Delete the models in the SpectrumDirectory\server\modules\hub\db directory.
      Warning: Do not delete the models from the master server.
    2. On one of the non-master servers, run the Spectrum™ Technology Platform installer to upgrade it to the new version of Spectrum™ Technology Platform.
    3. After the upgrade process finishes, the Spectrum™ Technology Platform server starts automatically. Wait for the server to finish starting completely, then stop the server.
    4. Open the file SpectrumFolder\server\modules\hub\hub.properties in an editor and set the property hub.neo4j.database.type to ha:
      hub.neo4j.database.type=ha
    5. Compare any properties files that you backed up to the installed files and make any necessary changes. Do not overwrite new files with old files because new files may contain properties that old files do not.
    6. Copy the SpectrumFolder\server\modules\hub\db\model.* directories from the master server to the non-master server.
    7. Repeat these steps for each non-master server in the cluster.
  11. Start the cluster.
    1. Start the master server, followed by non-master servers.
    2. Ensure that each server in the cluster is functioning. Check SpectrumFolder\server\app\repository\wrapper.log for any errors.
    3. While directly connected to the master server (bypassing the load balancer), open each model, one at a time, and inspect the wrapper.log file for errors.