Examples of Hive

org.apache.hadoop.hive.ql.metadata.Hive
The Hive class contains information about this instance of Hive. An instance of Hive represents a set of data in a file system (usually HDFS) organized for easy query processing
org.apache.wicket.security.hive.Hive
A Hive contains the security policy for the system, much like the {@link Policy} in JAAS does. However where the Policy only has a singleinstance for each virtual machine, multiple Hives are allowed per vm. Allowing for instance multiple web applications to each have there own security policies or share one. In effect making a hive a codebased policy. The {@link HiveMind} keeps a registry of hives based on keys, everyone withthe right key can access that hive, this allows you to choose if multiple (different) webapps all share the same same hive or let each instance of an application have there own hive. The key (sometimes referred to as the "queen", to stick with wasp terminology ;)) can be any object. A good choice would be the application itself or if it is a web application the contextname could be used. @author marrink
org.hivedb.Hive
@author Kevin Kelm (kkelm@fortress-consulting.com) @author Andy Likuski (alikuski@cafepress.com) @author Britt Crawford (bcrawford@cafepress.com)
The facade for all fundamental CRUD operations on the Hive directories and Hive metadata.
org.wicketstuff.security.hive.Hive
A Hive contains the security policy for the system, much like the {@link Policy} in JAAS does.However where the Policy only has a single instance for each virtual machine, multiple Hives are allowed per vm. Allowing for instance multiple web applications to each have there own security policies or share one. In effect making a hive a codebased policy. The {@link HiveMind} keeps aregistry of hives based on keys, everyone with the right key can access that hive, this allows you to choose if multiple (different) webapps all share the same same hive or let each instance of an application have there own hive. The key (sometimes referred to as the "queen", to stick with wasp terminology ;)) can be any object. A good choice would be the application itself or if it is a web application the contextname could be used. @author marrink
randomevents.hive.Hive
@author Kirill

Examples of org.apache.hadoop.hive.ql.metadata.Hive


  @Override
  public int execute(DriverContext driverContext) {


    // Create the db
    Hive db;
    try {
      db = Hive.get(conf);


      CreateDatabaseDesc createDatabaseDesc = work.getCreateDatabaseDesc();
      if (null != createDatabaseDesc) {

View Full Code Here

Examples of org.apache.hadoop.hive.ql.metadata.Hive

  private List<String> getTableData(String table, String database) throws Exception {
    HiveConf conf = new HiveConf();
    conf.addResource("hive-site.xml");
    ArrayList<String> results = new ArrayList<String>();
    ArrayList<String> temp = new ArrayList<String>();
    Hive hive = Hive.get(conf);
    org.apache.hadoop.hive.ql.metadata.Table tbl = hive.getTable(database, table);
    FetchWork work;
    if (!tbl.getPartCols().isEmpty()) {
      List<Partition> partitions = hive.getPartitions(tbl);
      List<PartitionDesc> partDesc = new ArrayList<PartitionDesc>();
      List<String> partLocs = new ArrayList<String>();
      for (Partition part : partitions) {
        partLocs.add(part.getLocation());
        partDesc.add(Utilities.getPartitionDesc(part));

View Full Code Here

Examples of org.apache.hadoop.hive.ql.metadata.Hive


  @Override
  public ASTNode preAnalyze(HiveSemanticAnalyzerHookContext context,
                ASTNode ast) throws SemanticException {


    Hive db;
    try {
      db = context.getHive();
    } catch (HiveException e) {
      throw new SemanticException(
        "Couldn't get Hive DB instance in semantic analysis phase.",
        e);
    }


    // Analyze and create tbl properties object
    int numCh = ast.getChildCount();


    String inputFormat = null, outputFormat = null;
    tableName = BaseSemanticAnalyzer.getUnescapedName((ASTNode) ast
      .getChild(0));
    boolean likeTable = false;


    for (int num = 1; num < numCh; num++) {
      ASTNode child = (ASTNode) ast.getChild(num);


      switch (child.getToken().getType()) {


      case HiveParser.TOK_QUERY: // CTAS
        throw new SemanticException(
          "Operation not supported. Create table as " +
            "Select is not a valid operation.");


      case HiveParser.TOK_TABLEBUCKETS:
        break;


      case HiveParser.TOK_TBLSEQUENCEFILE:
        inputFormat = HCatConstants.SEQUENCEFILE_INPUT;
        outputFormat = HCatConstants.SEQUENCEFILE_OUTPUT;
        break;


      case HiveParser.TOK_TBLTEXTFILE:
        inputFormat = org.apache.hadoop.mapred.TextInputFormat.class.getName();
        outputFormat = org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat.class.getName();


        break;


      case HiveParser.TOK_LIKETABLE:
        likeTable = true;
        break;


      case HiveParser.TOK_IFNOTEXISTS:
        try {
          List<String> tables = db.getTablesByPattern(tableName);
          if (tables != null && tables.size() > 0) { // table
            // exists
            return ast;
          }
        } catch (HiveException e) {

View Full Code Here

Examples of org.apache.hadoop.hive.ql.metadata.Hive

  private List<String> getTableData(String table, String database) throws Exception {
    HiveConf conf = new HiveConf();
    conf.addResource("hive-site.xml");
    ArrayList<String> results = new ArrayList<String>();
    ArrayList<String> temp = new ArrayList<String>();
    Hive hive = Hive.get(conf);
    org.apache.hadoop.hive.ql.metadata.Table tbl = hive.getTable(database, table);
    FetchWork work;
    if (!tbl.getPartCols().isEmpty()) {
      List<Partition> partitions = hive.getPartitions(tbl);
      List<PartitionDesc> partDesc = new ArrayList<PartitionDesc>();
      List<String> partLocs = new ArrayList<String>();
      for (Partition part : partitions) {
        partLocs.add(part.getLocation());
        partDesc.add(Utilities.getPartitionDesc(part));

View Full Code Here

Examples of org.apache.hadoop.hive.ql.metadata.Hive

  private static HiveLockObject getLockObject(HiveConf conf, String path,
    HiveLockMode mode, HiveLockObjectData data,
    String parent, boolean verifyTablePartition)
      throws LockException {
    try {
      Hive db = Hive.get(conf);
      int indx = path.lastIndexOf("LOCK-" + mode.toString());
      String objName = path.substring(("/" + parent + "/").length(), indx-1);
      String[] names = objName.split("/");


      if (names.length < 2) {
        return null;
      }


      if (!verifyTablePartition) {
        return new HiveLockObject(names, data);
      }


      // do not throw exception if table does not exist
      Table tab = db.getTable(names[0], names[1], false);
      if (tab == null) {
        return null;
      }


      if (names.length == 2) {
        return new HiveLockObject(tab, data);
      }


      Map<String, String> partSpec = new HashMap<String, String>();
      for (indx = 2; indx < names.length; indx++) {
        String[] partVals = names[indx].split("=");
        partSpec.put(partVals[0], partVals[1]);
      }


      Partition partn;
      try {
        partn = db.getPartition(tab, partSpec, false);
      } catch (HiveException e) {
        partn = null;
      }


      if (partn == null) {

View Full Code Here

Examples of org.apache.hadoop.hive.ql.metadata.Hive

   */
  public static Set<Partition> checkPartitionsCoveredByIndex(TableScanOperator tableScan,
      ParseContext pctx,
      Map<Table, List<Index>> indexes)
    throws HiveException {
    Hive hive = Hive.get(pctx.getConf());
    // make sure each partition exists on the index table
    PrunedPartitionList queryPartitionList = pctx.getOpToPartList().get(tableScan);
    Set<Partition> queryPartitions = queryPartitionList.getPartitions();
    if (queryPartitions == null || queryPartitions.isEmpty()) {
      return null;

View Full Code Here

Examples of org.apache.hadoop.hive.ql.metadata.Hive


  @Override
  public ASTNode preAnalyze(HiveSemanticAnalyzerHookContext context,
                ASTNode ast) throws SemanticException {


    Hive db;
    try {
      db = context.getHive();
    } catch (HiveException e) {
      throw new SemanticException(
        "Couldn't get Hive DB instance in semantic analysis phase.",
        e);
    }


    // Analyze and create tbl properties object
    int numCh = ast.getChildCount();


    String inputFormat = null, outputFormat = null;
    tableName = BaseSemanticAnalyzer.getUnescapedName((ASTNode) ast
      .getChild(0));
    boolean likeTable = false;


    for (int num = 1; num < numCh; num++) {
      ASTNode child = (ASTNode) ast.getChild(num);


      switch (child.getToken().getType()) {


      case HiveParser.TOK_QUERY: // CTAS
        throw new SemanticException(
          "Operation not supported. Create table as " +
            "Select is not a valid operation.");


      case HiveParser.TOK_TABLEBUCKETS:
        break;


      case HiveParser.TOK_TBLSEQUENCEFILE:
        inputFormat = HCatConstants.SEQUENCEFILE_INPUT;
        outputFormat = HCatConstants.SEQUENCEFILE_OUTPUT;
        break;


      case HiveParser.TOK_TBLTEXTFILE:
        inputFormat = org.apache.hadoop.mapred.TextInputFormat.class.getName();
        outputFormat = org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat.class.getName();


        break;


      case HiveParser.TOK_LIKETABLE:
        likeTable = true;
        break;


      case HiveParser.TOK_IFNOTEXISTS:
        try {
          List<String> tables = db.getTablesByPattern(tableName);
          if (tables != null && tables.size() > 0) { // table
            // exists
            return ast;
          }
        } catch (HiveException e) {

View Full Code Here

Examples of org.apache.hadoop.hive.ql.metadata.Hive

    throws HiveException, AuthorizationException {
    HashSet<ReadEntity> inputs = sem.getInputs();
    HashSet<WriteEntity> outputs = sem.getOutputs();
    SessionState ss = SessionState.get();
    HiveOperation op = ss.getHiveOperation();
    Hive db = sem.getDb();
    if (op != null) {
      if (op.equals(HiveOperation.CREATETABLE_AS_SELECT)
          || op.equals(HiveOperation.CREATETABLE)) {
        ss.getAuthorizer().authorize(
            db.getDatabase(SessionState.get().getCurrentDatabase()), null,
            HiveOperation.CREATETABLE_AS_SELECT.getOutputRequiredPrivileges());
      } else {
        if (op.equals(HiveOperation.IMPORT)) {
          ImportSemanticAnalyzer isa = (ImportSemanticAnalyzer) sem;
          if (!isa.existsTable()) {
            ss.getAuthorizer().authorize(
                db.getDatabase(SessionState.get().getCurrentDatabase()), null,
                HiveOperation.CREATETABLE_AS_SELECT.getOutputRequiredPrivileges());
          }
        }
      }
      if (outputs != null && outputs.size() > 0) {
        for (WriteEntity write : outputs) {


          if (write.getType() == WriteEntity.Type.PARTITION) {
            Partition part = db.getPartition(write.getTable(), write
                .getPartition().getSpec(), false);
            if (part != null) {
              ss.getAuthorizer().authorize(write.getPartition(), null,
                      op.getOutputRequiredPrivileges());
              continue;

View Full Code Here

Examples of org.apache.hadoop.hive.ql.metadata.Hive


  @Override
  public ASTNode preAnalyze(HiveSemanticAnalyzerHookContext context, ASTNode ast)
  throws SemanticException {


    Hive db;
    try {
      db = context.getHive();
    } catch (HiveException e) {
      throw new SemanticException("Couldn't get Hive DB instance in semantic analysis phase.", e);
    }


    // Analyze and create tbl properties object
    int numCh = ast.getChildCount();


    databaseName = BaseSemanticAnalyzer.getUnescapedName((ASTNode)ast.getChild(0));


    for (int num = 1; num < numCh; num++) {
      ASTNode child = (ASTNode) ast.getChild(num);


      switch (child.getToken().getType()) {


      case HiveParser.TOK_QUERY: // CTAS
        throw new SemanticException("Operation not supported. Create db as Select is not a valid operation.");


      case HiveParser.TOK_IFNOTEXISTS:
        try {
          List<String> dbs = db.getDatabasesByPattern(databaseName);
          if (dbs != null && dbs.size() > 0) { // db exists
            return ast;
          }
        } catch (HiveException e) {
          throw new SemanticException(e);

View Full Code Here

Examples of org.apache.hadoop.hive.ql.metadata.Hive


  @Override
  public ASTNode preAnalyze(HiveSemanticAnalyzerHookContext context, ASTNode ast)
  throws SemanticException {


    Hive db;
    try {
      db = context.getHive();
    } catch (HiveException e) {
      throw new SemanticException("Couldn't get Hive DB instance in semantic analysis phase.", e);
    }


    // Analyze and create tbl properties object
    int numCh = ast.getChildCount();


    String inputFormat = null, outputFormat = null;
    tableName = BaseSemanticAnalyzer.getUnescapedName((ASTNode)ast.getChild(0));


    for (int num = 1; num < numCh; num++) {
      ASTNode child = (ASTNode) ast.getChild(num);


      switch (child.getToken().getType()) {


      case HiveParser.TOK_QUERY: // CTAS
        throw new SemanticException("Operation not supported. Create table as Select is not a valid operation.");


      case HiveParser.TOK_TABLEBUCKETS:
        throw new SemanticException("Operation not supported. HCatalog doesn't allow Clustered By in create table.");


      case HiveParser.TOK_TBLSEQUENCEFILE:
        throw new SemanticException("Operation not supported. HCatalog doesn't support Sequence File by default yet. " +
        "You may specify it through INPUT/OUTPUT storage drivers.");


      case HiveParser.TOK_TBLTEXTFILE:
        throw new SemanticException("Operation not supported. HCatalog doesn't support Text File by default yet. " +
        "You may specify it through INPUT/OUTPUT storage drivers.");


      case HiveParser.TOK_LIKETABLE:


        String likeTableName;
        if (child.getChildCount() > 0 && (likeTableName = BaseSemanticAnalyzer.getUnescapedName((ASTNode)ast.getChild(0))) != null) {


          throw new SemanticException("Operation not supported. CREATE TABLE LIKE is not supported.");
//          Map<String, String> tblProps;
//          try {
//            tblProps = db.getTable(MetaStoreUtils.DEFAULT_DATABASE_NAME, likeTableName).getParameters();
//          } catch (HiveException he) {
//            throw new SemanticException(he);
//          }
//          if(!(tblProps.containsKey(InitializeInput.HOWL_ISD_CLASS) && tblProps.containsKey(InitializeInput.HOWL_OSD_CLASS))){
//            throw new SemanticException("Operation not supported. Table "+likeTableName+" should have been created through HCat. Seems like its not.");
//          }
//          return ast;
        }
        break;


      case HiveParser.TOK_IFNOTEXISTS:
        try {
          List<String> tables = db.getTablesByPattern(tableName);
          if (tables != null && tables.size() > 0) { // table exists
            return ast;
          }
        } catch (HiveException e) {
          throw new SemanticException(e);

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.