Examples of QNodeHandler

com.splout.db.qnode.QNodeHandler
Implements the business logic for the {@link QNode}.
The QNode is the most complex and delicate part of Splout. Among its responsabilities are:
- Handling deploys asynchronously: One QNode will lead a deployment. It will put a flag in ZooKeeper and trigger an asynchronous deploy to all involved DNodes. Then, it has to finalize the deploy properly when all DNodes are ready. This is handled by the {@link Deployer} module.
- Performing queries and multiqueries. This is handled by the {@link Querier} module.
- Handling rollbacks: Rollbacks are easy since we just need to change the version in ZooKeeper (DNodes already have the data for past version in disk). The number of versions that are saved in the system per tablespace can be configured (see {@link QNodeProperties}).
For convenience, there is some in-memory state grabbed from ZooKeeper in {@link QNodeHandlerContext}. This state is passed through all modules ( {@link Deployer} and such). Care has to be taken to have consistent in-memory state, forthat it is important to handle ZooKeeper events properly and be notified always on the paths that we are interested in.
One of the important business logic parts of this class is to synchronize the versions in ZooKeeper. Because we only want to keep a certain amount of versions, the QNodes have to check for this and remove stalled versions if needed. Then, DNodes will receive a notification and they will be able to delete the old data from disk.
The QNode returns JSON strings for all of its methods. The beans that are serialized are indicated in the documentation.

Examples of com.splout.db.qnode.QNodeHandler

  public void runForever() throws Exception {
    SploutConfiguration config = SploutConfiguration.getTestConfig();
    DNode dnode = new DNode(config, new DNodeHandler());
    dnode.init();
    QNode qnode = new QNode();
    qnode.start(config, new QNodeHandler());
  }

View Full Code Here

Examples of com.splout.db.qnode.QNodeHandler

    TestUtils.cleanUpTmpFolders(this.getClass().getName(), 2);
  }
  
  @Test
  public void testSimpleChange() throws Throwable {
    final QNodeHandler handler = new QNodeHandler();
    SploutConfiguration config = SploutConfiguration.getTestConfig();
    try {


      HazelcastInstance hz = Hazelcast.newHazelcastInstance(HazelcastConfigBuilder.build(config));
      CoordinationStructures coord = new CoordinationStructures(hz);
      /*
       * Create 5 successful versions. current-version will be set to last one.
       */
      
      // TODO
      
//      coord.getTablespaces().put(new TablespaceVersion("t1", 0l), new Tablespace(null, null, 0l, 0l));
//      coord.getTablespaces().put(new TablespaceVersion("t1", 1l), new Tablespace(null, null, 1l, 0l));
//      coord.getTablespaces().put(new TablespaceVersion("t1", 2l), new Tablespace(null, null, 2l, 0l));
//      coord.getTablespaces().put(new TablespaceVersion("t1", 3l), new Tablespace(null, null, 3l, 0l));
//      coord.getTablespaces().put(new TablespaceVersion("t1", 4l), new Tablespace(null, null, 4l, 0l));


      Map<String, Long> versionsBeingServed = new HashMap<String, Long>();
      versionsBeingServed.put("t1", 4l);
      coord.getVersionsBeingServed().put(CoordinationStructures.VERSIONS_BEING_SERVED, versionsBeingServed);


      handler.init(config);
      
      List<SwitchVersionRequest> rRequest = new ArrayList<SwitchVersionRequest>();
      SwitchVersionRequest theRequest = new SwitchVersionRequest("t1", 3);
      rRequest.add(theRequest);
      handler.rollback(rRequest);
      
      new TestUtils.NotWaitingForeverCondition() {
        @Override
        public boolean endCondition() {
          return handler.getContext().getCurrentVersionsMap().get("t1") != null &&
              handler.getContext().getCurrentVersionsMap().get("t1") == 3l;
        }
      }.waitAtMost(5000);
      
      assertEquals(3l, (long)coord.getCopyVersionsBeingServed().get("t1"));
      
      /*
       * Changing our mind : now back to previous version 4
       */
      theRequest = new SwitchVersionRequest("t1", 4);
      rRequest = new ArrayList<SwitchVersionRequest>();
      rRequest.add(theRequest);
      handler.rollback(rRequest);
            
      new TestUtils.NotWaitingForeverCondition() {
        @Override
        public boolean endCondition() {
          return handler.getContext().getCurrentVersionsMap().get("t1") != null &&
              handler.getContext().getCurrentVersionsMap().get("t1") == 4l;
        }
      }.waitAtMost(5000);
      
      assertEquals(4l, (long)coord.getCopyVersionsBeingServed().get("t1"));
    } finally {
      handler.close();
      Hazelcast.shutdownAll();
    }
  }

View Full Code Here

Examples of com.splout.db.qnode.QNodeHandler

  }
  
  @Test
  public void testMultiRollback() throws Throwable {
    // Here we will have more than one tablespace and we will rollback them all
    final QNodeHandler handler = new QNodeHandler();
    SploutConfiguration config = SploutConfiguration.getTestConfig();
    try {
      HazelcastInstance hz = Hazelcast.newHazelcastInstance(HazelcastConfigBuilder.build(config));
      CoordinationStructures coord = new CoordinationStructures(hz);
      
      /*
       * Create 5 successful versions. current-version will be set to last one.
       */
      
      // TODO 
      
//      for(int i = 0; i < 5; i++) {
//        coord.getTablespaces().put(new TablespaceVersion("t1", i), new Tablespace(null, null, i, 0l));
//        coord.getTablespaces().put(new TablespaceVersion("t2", i), new Tablespace(null, null, i, 0l));
//        coord.getTablespaces().put(new TablespaceVersion("t3", i), new Tablespace(null, null, i, 0l));
//      }
      
      // T1 -> current version 4
      // T2 -> current version 3
      // T3 -> current version 2
      Map<String, Long> versionsBeingServed = new HashMap<String, Long>();
      versionsBeingServed.put("t1", 4l);
      versionsBeingServed.put("t2", 3l);
      versionsBeingServed.put("t3", 2l);
      coord.getVersionsBeingServed().put(CoordinationStructures.VERSIONS_BEING_SERVED, versionsBeingServed);
      
      handler.init(config);
      
      List<SwitchVersionRequest> rRequest = new ArrayList<SwitchVersionRequest>();


      // T1 -> rollback to version 2
      // T2 -> rollback to version 1
      // T3 -> rollback to version 0
      SwitchVersionRequest theRequest = new SwitchVersionRequest("t1", 2l);
      rRequest.add(theRequest);
      theRequest = new SwitchVersionRequest("t2", 1l);
      rRequest.add(theRequest);
      theRequest = new SwitchVersionRequest("t3", 0l);
      rRequest.add(theRequest);
      
      handler.rollback(rRequest);
      
      new TestUtils.NotWaitingForeverCondition() {
        @Override
        public boolean endCondition() {
          return
              handler.getContext().getCurrentVersionsMap().get("t1") != null &&
              handler.getContext().getCurrentVersionsMap().get("t1") == 2l;
        }
      }.waitAtMost(5000);


      assertEquals(2l, (long)coord.getCopyVersionsBeingServed().get("t1"));
      assertEquals(1l, (long)coord.getCopyVersionsBeingServed().get("t2"));
      assertEquals(0l, (long)coord.getCopyVersionsBeingServed().get("t3"));
      
      assertEquals(2l, (long)handler.getContext().getCurrentVersionsMap().get("t1"));
      assertEquals(1l, (long)handler.getContext().getCurrentVersionsMap().get("t2"));
      assertEquals(0l, (long)handler.getContext().getCurrentVersionsMap().get("t3"));
      
    } finally {
      handler.close();
      Hazelcast.shutdownAll();
    }
  }

View Full Code Here

Examples of com.splout.db.qnode.QNodeHandler

          for(int i = 0; i < N_QNODES; i++) {
            List<String> dNodeList = clients[i].dNodeList();
            if(dNodeList.size() != 3) {
              return false;
            }
            QNodeHandler handler = (QNodeHandler)getqNodes().get(i).getHandler();
            for(String dnode: dNodeList) {
              if(handler.getContext().getThriftClientCache().get(dnode) == null) {
                return false;
              }
            }
          }
          return true;

View Full Code Here

Examples of com.splout.db.qnode.QNodeHandler

      config.setProperty(FetcherProperties.TEMP_DIR, config.getString(FetcherProperties.TEMP_DIR) + "-" + i);
      DNode dnode = new DNode(config, new DNodeHandler());
      dnode.init();
    }
    QNode qnode = new QNode();
    qnode.start(config, new QNodeHandler());
  }

View Full Code Here

Examples of com.splout.db.qnode.QNodeHandler

      // Assert that there is only MAX_VERSIONS versions of the tablespace (due to old version cleanup)
      new TestUtils.NotWaitingForeverCondition() {


        @Override
        public boolean endCondition() {
          QNodeHandler handler = (QNodeHandler) qNodes.get(0).getHandler();
          int seenVersions = 0;
          for(Map.Entry<TablespaceVersion, Tablespace> tablespaceVersion : handler.getContext()
              .getTablespaceVersionsMap().entrySet()) {
            if(tablespaceVersion.getKey().getTablespace().equals(TABLESPACE)) {
              seenVersions++;
            }
          }

View Full Code Here

Examples of com.splout.db.qnode.QNodeHandler

    testConfig.setProperty(DNodeProperties.HANDLE_TEST_COMMANDS, true);
    final DNode dnode = new DNode(testConfig, new DNodeHandler());
    dnode.init();
    
    final QNode qnode = new QNode();
    qnode.start(testConfig, new QNodeHandler());


    // Wait until the QNode has 1 Dnode in the list
    waitUntilThereAreThatManyDNodes(qnode, 1);
    
    dnode.testCommand(TestCommands.SHUTDOWN.toString());

View Full Code Here

Examples of com.splout.db.qnode.QNodeHandler

    /*
     * Create an ensemble of QNodes
     */
    for(int i = 0; i < nQnodes; i++) {
      SploutConfiguration qNodeConfig = SploutConfiguration.getTestConfig();
      QNode qnode = TestUtils.getTestQNode(qNodeConfig, new QNodeHandler());
      qNodes.add(qnode);
    }


    /*
     * Create an ensemble of DNodes

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.