메뉴 건너뛰기

Cloudera, BigData, Semantic IoT, Hadoop, NoSQL

Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.


hbase Hbase Shell 명령 정리

구퍼 2013.04.01 13:25 조회 수 : 3300

$ ${HBASE_HOME}/bin/hbase shell

You'll be presented with a prompt like the following:

HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.2.0-dev, r670701, Mon Jun 23 17:26:36 PDT 2008
hbase(main):001:0>

Type 'help' followed by a return to get a listing of commands.

Commands

hbase(main):001:0> help
HBASE SHELL COMMANDS:
 alter     Alter column family schema;  pass table name and a dictionary
           specifying new column family schema. Dictionaries are described
           below in the GENERAL NOTES section.  Dictionary must include name
           of column family to alter.  For example,

           To change or add the 'f1' column family in table 't1' from defaults
           to instead keep a maximum of 5 cell VERSIONS, do:
           hbase> alter 't1', {NAME => 'f1', VERSIONS => 5}

           To delete the 'f1' column family in table 't1', do:
           hbase> alter 't1', {NAME => 'f1', METHOD => 'delete'}

           You can also change table-scope attributes like MAX_FILESIZE
           MEMSTORE_FLUSHSIZE and READONLY.

           For example, to change the max size of a family to 128MB, do:
           hbase> alter 't1', {METHOD => 'table_att', MAX_FILESIZE => '134217728'}

 count     Count the number of rows in a table. This operation may take a LONG
           time (Run '$HADOOP_HOME/bin/hadoop jar hbase.jar rowcount' to run a
           counting mapreduce job). Current count is shown every 1000 rows by
           default. Count interval may be optionally specified. Examples:

           hbase> count 't1'
           hbase> count 't1', 100000

 create    Create table; pass table name, a dictionary of specifications per
           column family, and optionally a dictionary of table configuration.
           Dictionaries are described below in the GENERAL NOTES section.
           Examples:

           hbase> create 't1', {NAME => 'f1', VERSIONS => 5}
           hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'}
           hbase> # The above in shorthand would be the following:
           hbase> create 't1', 'f1', 'f2', 'f3'
           hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000, 
             BLOCKCACHE => true}

 describe  Describe the named table: e.g. "hbase> describe 't1'"

 delete    Put a delete cell value at specified table/row/column and optionally
           timestamp coordinates.  Deletes must match the deleted cell's
           coordinates exactly.  When scanning, a delete cell suppresses older
           versions. Takes arguments like the 'put' command described below

 deleteall Delete all cells in a given row; pass a table name, row, and optionally
           a column and timestamp

 disable   Disable the named table: e.g. "hbase> disable 't1'"

 drop      Drop the named table. Table must first be disabled. If table has
           more than one region, run a major compaction on .META.:

           hbase> major_compact ".META."

 enable    Enable the named table

 exists    Does the named table exist? e.g. "hbase> exists 't1'"

 exit      Type "hbase> exit" to leave the HBase Shell

 get       Get row or cell contents; pass table name, row, and optionally
           a dictionary of column(s), timestamp and versions.  Examples:

           hbase> get 't1', 'r1'
           hbase> get 't1', 'r1', {COLUMN => 'c1'}
           hbase> get 't1', 'r1', {COLUMN => ['c1', 'c2', 'c3']}
           hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1}
           hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1, 
             VERSIONS => 4}

 list      List all tables in hbase

 put       Put a cell 'value' at specified table/row/column and optionally
           timestamp coordinates.  To put a cell value into table 't1' at
           row 'r1' under column 'c1' marked with the time 'ts1', do:

           hbase> put 't1', 'r1', 'c1', 'value', ts1

 tools     Listing of hbase surgery tools

 scan      Scan a table; pass table name and optionally a dictionary of scanner
           specifications.  Scanner specifications may include one or more of
           the following: LIMIT, STARTROW, STOPROW, TIMESTAMP, or COLUMNS.  If
           no columns are specified, all columns will be scanned.  To scan all
           members of a column family, leave the qualifier empty as in
           'col_family:'.  Examples:

           hbase> scan '.META.'
           hbase> scan '.META.', {COLUMNS => 'info:regioninfo'}
           hbase> scan 't1', {COLUMNS => ['c1', 'c2'], LIMIT => 10, 
             STARTROW => 'xyz'}

           For experts, there is an additional option -- CACHE_BLOCKS -- which
           switches block caching for the scanner on (true) or off (false).  By
           default it is enabled.  Examples:

           hbase> scan 't1', {COLUMNS => ['c1', 'c2'], CACHE_BLOCKS => false}

 status    Show cluster status. Can be 'summary', 'simple', or 'detailed'. The
           default is 'summary'. Examples:

           hbase> status
           hbase> status 'simple'
           hbase> status 'summary'
           hbase> status 'detailed'

 shutdown  Shut down the cluster.

 truncate  Disables, drops and recreates the specified table.

 version   Output this HBase version

GENERAL NOTES:
Quote all names in the hbase shell such as table and column names.  Don't
forget commas delimit command parameters.  Type <RETURN> after entering a
command to run it.  Dictionaries of configuration used in the creation and
alteration of tables are ruby Hashes. They look like this:

  {'key1' => 'value1', 'key2' => 'value2', ...}

They are opened and closed with curley-braces.  Key/values are delimited by
the '=>' character combination.  Usually keys are predefined constants such as
NAME, VERSIONS, COMPRESSION, etc.  Constants do not need to be quoted.  Type
'Object.constants' to see a (messy) list of all constants in the environment.

In case you are using binary keys or values and need to enter them into the
shell then use double-quotes to make use of hexadecimal or octal notations,
for example:

  hbase> get 't1', "keyx03x3fxcd"
  hbase> get 't1', "key-03-3-1"
  hbase> put 't1', "testxefxff", 'f1:', "x01x33x40"

Using the double-quote notation you can directly use the values output by the
shell for example during a "scan" call.

This HBase shell is the JRuby IRB with the above HBase-specific commands added.
For more on the HBase Shell, see http://wiki.apache.org/hadoop/Hbase/Shell

General Notes

GENERAL NOTES: Quote all names in the hbase shell such as table and column names. Don't forget commas delimit command parameters. Type <RETURN> after entering a command to run it. Dictionaries of configuration used in the creation and alteration of tables are ruby Hashes. They look like this:

  • {'key1' => 'value1', 'key2' => 'value2', ...}

They are opened and closed with curley-braces. Key/values are delimited by the '=>' character combination. Usually keys are predefined constants such as NAME, VERSIONS, COMPRESSION, etc. Constants do not need to be quoted. Type 'Object.constants' to see a (messy) list of all constants in the environment.

This HBase shell is the JRuby IRB with the above HBase-specific commands added.

Anatomy Lesson

The HBase Shell is a ruby script, ${HBASE_HOME}/bin/hirb.rb, that is passed to the JRuby class org.jruby.Main (See ${HBASE_HOME}/bin/shell and search for org.jruby.Main to see how its done).

The hirb.rb' script defines a set of hbase methods (get, set, scan, etc.), sets some environment variables, imports a few hbase-particular modules -- a results Formatter and a ruby wrapper around theHBaseAdmin and HTable java classes that can be found at ${HBASE_HOME}/bin/HBase.rb -- and then starts up a subclass of IRB, named HIRB (IRB is the name of the interactive Ruby interpreter; for an untainted irb experience, type 'irb' on a system that has ruby installed). The subclassing is done to alter slightly some of the usual IRB behaviors. For instance, every command invocation returns a result and the result is printed to the terminal even if the result is nil. The subclass intercepts nil emissions and just suppresses them.

Anything you can do in irb, or at least in its JRuby equivalent, jirb, you should be able to do in HBase Shell.

Scripting

You can pass scripts to the HBase Shell by doing the following:

durruti:~ stack$ ${HBASE_HOME}/bin/hbase shell PATH_TO_SCRIPT

Your script can lean on the methods provided by the HBase Shell.

For more control, you may prefer manipulating HTable and HBaseAdmin methods directly. For example, if you would do your own formatting or you are storing structures that are mangled when manipulated in HBase Shell. In this case, you might pass your scripts directly to JRuby by doing:

durruti:~ stack$ ${HBASE_HOME}/bin/hbase org.jruby.Main PATH_TO_SCRIPT

For examples writing straight ruby manhandling hbase client instances, see the scripts on this page: Hbase/JRuby (Ignore the sections that have you fetching the jruby jar and the prefacing your scripts with the '#!' bang magic).

Useful Tricks

irbrc

Create an .irbrc file for yourself in your home directory. Add HBase Shell customizations. A useful one is command history:

durruti:~ stack$ more .irbrc
require 'irb/ext/save-history'
IRB.conf[:SAVE_HISTORY] = 100
IRB.conf[:HISTORY_FILE] = "#{ENV['HOME']}/.irb-save-history"

Log date to timestamp

To convert the date '08/08/16 20:56:29' from an hbase log into a timestamp, do:

hbase(main):021:0> import java.text.SimpleDateFormat
hbase(main):022:0> import java.text.ParsePosition
hbase(main):023:0> SimpleDateFormat.new("yy/MM/dd HH:mm:ss").parse("08/08/16 20:56:29", ParsePosition.new(0)).getTime() => 1218920189000

To go the other direction, do:

hbase(main):021:0> import java.util.Date
hbase(main):022:0> Date.new(1218920189000).toString() => "Sat Aug 16 20:56:29 UTC 2008"

To output in a format that is exactly like hbase log format is a pain messing with SimpleDateFormat.


번호 제목 날짜 조회 수
721 [CDP7.1.7]Oozie job에서 ERROR: Kudu error(s) reported, first error: Timed out: Failed to write batch of 774 ops to tablet 8003f9a064bf4be5890a178439b2ba91가 발생하면서 쿼리가 실패하는 경우 2024.01.05 118
720 [Hadoop Encryption] Encryption Zone에 생성된 table에 Hue에서 insert 수행시 User:hdfs not allowed to do 'DECRYPT_EEK' ON 'testkey' 오류 2023.11.01 120
719 hadoop에서 yarn jar ..를 이용하여 appliction을 실행하여 정상적으로 수행되었으나 yarn UI의 어플리케이션 목록에 나타나지 않는 문제 2017.05.02 122
718 5건의 triple data를 이용하여 특정 작업 폴더에서 작업하는 방법/절차 2016.06.16 123
717 magento2 설치후 초기화면이 깨지는 문제 file 2017.01.31 123
716 [kerberos]Kerberos HA구성 참고 페이지 2022.08.31 123
715 magento2 샘플데이타 설치 2017.01.31 124
714 [Kerberos]병렬 kinit 호출시 cache파일이 손상되어 Bad format in credentials cache 혹은 No credentials cache found 혹은 Internal credentials cache error 오류 발생시 2023.01.20 125
713 [oozie]Oozie WF수행시 단계별 ID넘버링 비교/설명 2022.03.23 126
712 [bitbucket] 2022년 3월 2일 부터 git 작업시 기존에 사용하던 비빌번호를 사용할 수 없도록 변경되었다. 2022.04.30 126
711 [oracle]10자리 timestamp값을 날짜로 변환하는 방법 2022.04.14 127
710 core 'gc_shard3_replica2' is already locked라는 오류가 발생할때 조치사항 2017.09.14 130
709 [CDP7.1.3]Ranger WebUI에서 Error! Connection refused: Please check the KMS provider URL and whether the Ranager KMS is running발생시 조치 방법 2023.06.07 131
708 [CDP7.1.7] oozie sqoop action으로 import혹은 export수행시 발생한 오류에 대한 자세한 로그 확인 하는 방법 2024.04.19 131
707 webid에서 google처럼 검색할 수 있도록 하는 프로그램 2017.05.16 133
706 Hadoop 2.7.x에서 사용할 수 있는 파일/디렉토리 관련 util성 클래스 파일 2017.09.28 133
705 [HA구성 이슈]oozie 2대를 L4로 HA구성했을때 발생하는 이슈 2023.01.17 133
704 S2RDF 테스트(벤치마크 테스트를 기준으로 python, scala소스가 만들어져서 기능은 파악되지 못함) [2] file 2016.05.27 135
703 Oracle NLOB type의 데이터를 import하는 경우 No Java type for SQL type 2011 for column rst와 같은 오류 발생시 조치사항 2022.01.14 135
702 vuestorefrontui.io를 이용한 front end project 생성하기 2022.02.06 135
위로