Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.

Cassandra+CumulusRDF cumulusRDF 1.0.1설치및 "KeyspaceCumulus" keyspace확인하기

총관리자 2016.04.15 17:39 조회 수 : 6245

1. Cassandra 1.2.11 다운로드/설치(cumulusRDF 1.0.1와 호환되며 테스트 된 버젼임)

 http://archive.apache.org/dist/cassandra/

*참고1 : https://www.gooper.com/ss/index.php?mid=bigdata&category=2803&document_srl=3110 (버젼이 다르지만 언급된 항목에 대한 설정은 같으므로 참고하여 설정해준다)

*참고2 : CumulusRDF 1.1.0이 Cassandra 1.2.X만 지원하므로 1.2.X를 다운받아 설치해야한다.

2. CumulusRDF 1.0.1 Web Application다운로드및 설치(직접 maven으로 빌드하면.. *.jar, war파일이 만들어지나 잘안됨(?))

https://github.com/cumulusrdf/cumulusrdf/wiki/Downloads

에서 March 11th 2014: CumulusRDF v1.0.1 war파일을 다운로드 받아서 WAS에 deploy한다.

(예, tomcat의 경우 webapps폴더 밑에 두면 파일명을 context명으로 자동설치된다)

* 참고1 : https://github.com/cumulusrdf/cumulusrdf/wiki

* 참고2 : http://xxx.xxx.xxx.43:8080/cumulusrdf-1.0.1/info에 접근하면 web페이지에서 query및 bulkupload를 할 수있다. 

3. CumulusRDF 1.0.1 CLI 툴 다운로드및 설치

  https://github.com/cumulusrdf/cumulusrdf/wiki/Downloads

에서 March 11th 2014: CumulusRDF v1.0.1 CLI jar를 다운로드 받아서 적절한 위치에 복사한다.

* 참고1 : https://github.com/cumulusrdf/cumulusrdf/wiki/CLI)

* 참고2 : dump, load, query, remove를 실행할 수 있는 jar파일임

4. 첨부된 1.0.0버젼의 CLI jar파일은 load할때 아래와 같이 사용한다.(이것은 thread개수를 지정할 수 있는데.. CumulusRDF 1.0.1 CLI등은 사용법이르며 일부기능이 지원되지 않음)

java -cp ./cumulusrdf-1.0.0-jar-with-dependencies.jar edu.kit.aifb.cumulus.cli.Main Load -i ./icbms_2016-04-15_15-10-16.nq -b 10000 -t 8

: 배치 10000, 쓰레드 8개로 nq파일을 업로드함

: 첨부파일을 이용할것

----------------------------첨부된 jar파일 사용시 가능한 옵션(Load, Dump, Query, Remove별로 다름)----------------------

-bash-4.1# java -cp ./cumulusrdf-1.0.0-jar-with-dependencies.jar edu.kit.aifb.cumulus.cli.Main Load -help

***ERROR: class org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -help

usage: parameters:

-b <arg> batch size - number of triples (default: 100)

-f <arg> format ('nt', 'nq' or 'xml') (default: 'nt')

-h print help

-i <arg> name of file to read, - for stdin (but then need to specify -x

option)

-k <arg> Cassandra keyspace (default KeyspaceCumulus)

-n <arg> Cassandra hosts as comma-separated list

('host1:port1,host2:port2,...') (default localhost:9160)

-r <arg> replication factor (default: 1)

-s <arg> storage layout to use (triple|quad) (needs to match webapp

configuration)

-t <arg> number of loading threads (defaults to min(1,|hosts|/1.5))

time elapsed 6 ms

-bash-4.1# java -cp ./cumulusrdf-1.0.0-jar-with-dependencies.jar edu.kit.aifb.cumulus.cli.Main Dump -help

***ERROR: class org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -help

usage: parameters:

-h print help

-k <arg> Cassandra keyspace (default KeyspaceCumulus)

-n <arg> Cassandra hosts as comma-separated list

('host1:port1,host2:port2,...') (default localhost:9160)

-o <arg> name of output file

time elapsed 6 ms

-bash-4.1# java -cp ./cumulusrdf-1.0.0-jar-with-dependencies.jar edu.kit.aifb.cumulus.cli.Main Query -help

***ERROR: class org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -help

usage: parameters:

-h print help

-k <arg> Cassandra keyspace (default KeyspaceCumulus)

-n <arg> Cassandra hosts as comma-separated list

('host1:port1,host2:port2,...') (default localhost:9160)

-q <arg> sparql query string

-s <arg> storage layout to use (triple|quad) (needs to match webapp

configuration)

time elapsed 10 ms

-bash-4.1# java -cp ./cumulusrdf-1.0.0-jar-with-dependencies.jar edu.kit.aifb.cumulus.cli.Main Remove -help

***ERROR: class org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -help

usage: parameters:

-h print help

-k <arg> Cassandra keyspace (default KeyspaceCumulus)

-n <arg> Cassandra hosts as comma-separated list

('host1:port1,host2:port2,...') (default localhost:9160)

-q <arg> sparql construct query string. all its bindings will be

removed.

-s <arg> storage layout to use (triple|quad) (needs to match webapp

configuration)

time elapsed 11 ms

--------------------------------cql.sh--------------

가. keyspace확인하기

cqlsh> select * from system.schema_keyspaces;

===>

keyspace_name | durable_writes | strategy_class | strategy_options

-----------------+----------------+---------------------------------------------+----------------------------

KeyspaceCumulus | True | org.apache.cassandra.locator.SimpleStrategy | {"replication_factor":"1"}

system | True | org.apache.cassandra.locator.LocalStrategy | {}

system_traces | True | org.apache.cassandra.locator.SimpleStrategy | {"replication_factor":"2"}

---------------

* keyspace목록 보기 : cqlsh>describe keyspaces;

* columnfamilies목록 보기 : describe columnfamilies;

* keyspace정보 보기 : describe keyspace "KeyspaceCumulus";

나. 사용할 keyspace지정하기

use "KeyspaceCumulus";

다. 테이블 목록 조회

describe tables;

*테이블 내용조회

select * from "DICT_P" limit 10;

라. KeyspaceCumulus가 사용하는 테이블 목록

TRUNCATE "DICT_P";

TRUNCATE "OSPC";

TRUNCATE "PREFIX_TO_NS";

TRUNCATE "SPOC";

TRUNCATE "DICT_P_REVERSE";

TRUNCATE "POSC";

TRUNCATE "SCHEMA_CLASSES";

TRUNCATE "SPO_RN_DT";

TRUNCATE "DICT_SO";

TRUNCATE "POS_RN_DT";

TRUNCATE "SCHEMA_D_PROPS";

TRUNCATE "SPO_RN_NUM";

TRUNCATE "DICT_SO_REVERSE";

TRUNCATE "POS_RN_NUM";

TRUNCATE "SCHEMA_O_PROPS";

TRUNCATE "counter";

-----------------------------------KeyspaceCumulus정보---------------------

cqlsh> describe keyspace "KeyspaceCumulus";

CREATE KEYSPACE "KeyspaceCumulus" WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};

USE "KeyspaceCumulus";

CREATE TABLE "DICT_P" (
key blob,
column1 blob,
value blob,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "DICT_P_REVERSE" (
key blob,
column1 blob,
value blob,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "DICT_SO" (
key blob,
column1 blob,
value blob,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "DICT_SO_REVERSE" (
key blob,
column1 blob,
value blob,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "OSPC" (
key blob,
column1 blob,
column2 blob,
column3 blob,
value blob,
PRIMARY KEY (key, column1, column2, column3)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "POSC" (
key blob,
column1 blob,
column2 blob,
column3 blob,
"03" blob,
PRIMARY KEY (key, column1, column2, column3)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE INDEX index_924575 ON "POSC" ("03");

CREATE TABLE "POS_RN_DT" (
key blob,
column1 bigint,
column2 blob,
value blob,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "POS_RN_NUM" (
key blob,
column1 double,
column2 blob,
value blob,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "PREFIX_TO_NS" (
key blob,
column1 blob,
value blob,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "SCHEMA_CLASSES" (
key blob,
column1 blob,
value blob,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "SCHEMA_D_PROPS" (
key blob,
column1 blob,
value blob,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "SCHEMA_O_PROPS" (
key blob,
column1 blob,
value blob,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "SPOC" (
key blob,
column1 blob,
column2 blob,
column3 blob,
value blob,
PRIMARY KEY (key, column1, column2, column3)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "SPO_RN_DT" (
key blob,
column1 bigint,
value blob,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE "SPO_RN_NUM" (
key blob,
column1 double,
value blob,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=0 AND
read_repair_chance=0.000000 AND
replicate_on_write='false' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

CREATE TABLE counter (
key text,
column1 text,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=1.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};

cumulusrdf-1.0.0-jar-with-dependencies.jar [File Size:13.40MB/Download:60]

이 게시물을

이 글의 추천인 목록 목록

번호	제목	날짜	조회 수
18	solrcloud에 solrdf1.1설치하고 테스트 하기	2016.04.22	687
»	cumulusRDF 1.0.1설치및 "KeyspaceCumulus" keyspace확인하기	2016.04.15	6245
16	System Properties Comparison Elasticsearch vs. Hive vs. Jena	2016.03.10	1055
15	TopBraid Composer에서 SPIN 사용법	2016.02.25	231
14	SPIN(SPARQL Inference Notation)이란..	2016.02.25	1007
13	SPARQL의 유형, SPARQL 만들기등에 대한 설명	2016.02.18	434
12	select와 group by결과 값이 없는경우의 리턴 값이 다름	2016.02.05	428
11	?a는 모두 표시하면서 ?b와 비교하여 ?a=?b는 표시하고 ?a!=?b 인경우는 ""로 구성된 결과 집합을 구하는 경우	2016.01.29	917
10	null 혹은 ""를 체크하는 방법	2016.01.27	773
9	sparql 1.1 BIND(if() as ?bind변수) 버그로 추정되는 문제점및 해결방안 -> select 문에 (if(,,) as ?bind변수)	2016.01.21	478
8	한번에 여러값 update하기	2016.01.13	567
7	문자열을 숫자(integer)로 casting하기	2016.01.13	1194
6	update(update와 delete->insert)사용시 주의/참고사항	2016.01.06	1007
5	fuseki에 update하는 방법(java api이용)및 주의 사항	2015.12.30	944
4	sparql 문법구조 설명	2015.12.09	878
3	protege 4.3 다운로드	2015.12.09	357
2	ontology, jena, sparql등 전반에 대한 설명및 예제를 제공하는 사이트	2015.12.08	553
1	sparql에서 concat에제	2015.11.27	599

쓰기 태그

첫 페이지 1 2 3 4 끝 페이지

Cloudera, BigData, Semantic IoT, Hadoop, NoSQL

Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.

Cassandra+CumulusRDF cumulusRDF 1.0.1설치및 "KeyspaceCumulus" keyspace확인하기

1. Cassandra 1.2.11 다운로드/설치(cumulusRDF 1.0.1와 호환되며 테스트 된 버젼임)

2. CumulusRDF 1.0.1 Web Application다운로드및 설치(직접 maven으로 빌드하면.. *.jar, war파일이 만들어지나 잘안됨(?))

3. CumulusRDF 1.0.1 CLI 툴 다운로드및 설치

4. 첨부된 1.0.0버젼의 CLI jar파일은 load할때 아래와 같이 사용한다.(이것은 thread개수를 지정할 수 있는데.. CumulusRDF 1.0.1 CLI등은 사용법이르며 일부기능이 지원되지 않음)

--------------------------------cql.sh--------------

댓글 0

Cloudera, BigData, Semantic IoT, Hadoop, NoSQL

Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.

Cassandra+CumulusRDF cumulusRDF 1.0.1설치및 "KeyspaceCumulus" keyspace확인하기

1. Cassandra 1.2.11 다운로드/설치(cumulusRDF 1.0.1와 호환되며 테스트 된 버젼임)

2. CumulusRDF 1.0.1 Web Application다운로드및 설치(직접 maven으로 빌드하면.. *.jar, war파일이 만들어지나 잘안됨(?))

3. CumulusRDF 1.0.1 CLI 툴 다운로드및 설치

4. 첨부된 1.0.0버젼의 CLI jar파일은 load할때 아래와 같이 사용한다.(이것은 thread개수를 지정할 수 있는데.. CumulusRDF 1.0.1 CLI등은 사용법이르며 일부기능이 지원되지 않음)

--------------------------------cql.sh--------------

댓글 0

LOGIN