Reindex content on a clustered IBM Connections deployment

Overview

When you reindex from scratch on a clustered IBM Connections deployment, reindexing takes place in the background while the system continues to service requests of the existing index. When reindexing completes, you then switch to the new index. The update process includes ripple restart of Search nodes.

This procedure uses the following three-node cluster as an example:

• Dmgr host: dmgrhost
• Search node1 host1
• Search node2: host2

Index location:/local/IBM/LotusConnections/data/local/search/index

Reindex cluster

1. Verify both search nodes have enough free disk space for seedlist files and a new index. We need at least three times the size of the current index:
   df -kh
   du -ksh /local/IBM/LotusConnections/data/local/search/index

2. Create a placeholder for the new index on node1:
   mkdir --parents /tmp/index.foo
   cd /tmp/index.foo
   mkdir ./backgroundCrawl ./backgroundFileExtraction ./backgroundIndex

3. Onde node1, start background indexing and crawling of all configured services:
   cd /local/IBM/WebSphere8/AppServer/profiles/Dmgr01/bin
   ./wsadmin.sh-lang jython
   execfile("searchAdmin.py")
   SearchService.startBackgroundIndex("/tmp/index.foo/backgroundCrawl", "/tmp/index.foo/backgroundFileExtraction", "/tmp/index.foo/backgroundIndex", "all_configured")

4. Monitor progress on node1's SystemOut.log for any warning or error messages. Example:
   CLFRW0577I: Search has started background crawling and indexing.
   Index will be built in /tmp/index.foo/backgroundIndex.

   When the process completes correctly, the following marker file is created:

   /tmp/index.foo/backgroundIndex/INDEX.READY

5. Copy the new index to node2:
   mkdir /tmp/index.foo
   scp -r root@host1:/tmp/index.foo/backgroundIndex /tmp/index.foo/

   This step does not replace the active index.

6. Prepare to replace the active index on the deployment manager node:

   1. Record all enabled indexing tasks:
      SearchService.listTasks()

   2. Disable all enabled indexing tasks for each enabled task:
      SearchService.disableTask("taskname")

   3. Verify that no tasks are still running. You should get an empty output from the following command:
      SearchService.listRunningTasks()

7. Replace the active index on node1:

   1. Stop WebSphere Application Server on Search node1.

   2. Move the original index out of active position:
      mv /local/IBM/LotusConnections/data/local/search/index/* /local/IBM/LotusConnections/data/local/search/index.previous

   3. Move the new index into active position:
      mv /tmp/index.foo/backgroundIndex/* /local/IBM/LotusConnections/data/local/search/index

   4. Start WebSphere Application Server on Search node1.

8. Replace the active index on node2:

   1. Stop WebSphere Application Server on Search node2.

   2. Move the original index out of active position:
      mv /local/IBM/LotusConnections/data/local/search/index/* /local/IBM/LotusConnections/data/local/search/index.previous

   3. Move the new index into active position:
      mv /tmp/index.foo/backgroundIndex/* /local/IBM/LotusConnections/data/local/search/index

   4. Start WebSphere Application Server on Search node2.

9. Resume indexing over the new index, take the following steps:

   1. On the deployment manager node, close and reopen wsadmin. Enter the following commands:
      cd /local/IBM/WebSphere8/AppServer/profiles/Dmgr01/bin
      ./wsadmin.sh -lang jython
      execfile("searchAdmin.py")

   2. Purge all traces of the previous index from the database:
      SearchService.notifyRestore("true")

   3. Enable all tasks, for each task you previously disabled:
      SearchService.enableTask("taskname");

10. Reindex People finder with the newly generated social graph:
    SearchService.addRunOncePeopleFinderSaNDIndexingTask()

Migration issue: Search

[11/10/14 17:42:12:874 EST] 00000159 AlarmListener E SCHD0063E: A task with ID 58 (ReplyToMailRetrieval) failed to run on Scheduler NewsScheduler (scheduler/news) because of an exception: com.ibm.websphere.scheduler.UserCalendarInvalid: SCHD0013E: The following error occurred while accessing the UserCalendar with JNDI Name com/ibm/websphere/scheduler/calendar/DefaultUserCalendarHome and specifier CRON : SCHD0137E: Unable to create EJB instance for UserCalendar: java.rmi.RemoteException: CORBA TRANSIENT 0x4942fe07 No; nested exception is:
org.omg.CORBA.TRANSIENT: initial and forwarded IOR inaccessible vmcid: IBM minor code: E07 completed: No SCHD0140I: EJB information: Host: myhost., Port: 9,101, J2EE component: SchedulerCalendars#Calendars.jar#DefaultUserCalendar..
[11/10/14 17:58:28:168 EST] 00000150 J2EEContext E ASYN9999E: Unexpected Exception Occurred: com.ibm.websphere.asynchbeans.SerialDeserialException: Exception while deserializing a saved service. Service=security. Unable to deserialize the Subjects in this Context, cause: Validation of LTPA token failed due to invalid keys or token type.

To fix, run clearScheduler.sql

See Switching from a stand-alone install to a cluster install causes problems with the Search scheduler

Parent topic:
Manage the Search index