version

The versions required for Spark and Java are as follow:

install

mvn clean scala:compile compile install.

1. maven dependency

1. <dependency>
2. <groupId>org.apache.iotdb</groupId>
3. <artifactId>spark-iotdb-connector</artifactId>
4. <version>0.10.0</version>
5. </dependency>

2. spark-shell user guide

1. spark-shell --jars spark-iotdb-connector-0.10.0.jar,iotdb-jdbc-0.10.0-jar-with-dependencies.jar
3. import org.apache.iotdb.spark.db._
5. val df = spark.read.format("org.apache.iotdb.spark.db").option("url","jdbc:iotdb://127.0.0.1:6667/").option("sql","select * from root").load
7. df.printSchema()
9. df.show()

如果要对RDD进行分区，可以执行以下操作

1. spark-shell --jars spark-iotdb-connector-0.10.0.jar,iotdb-jdbc-0.10.0-jar-with-dependencies.jar
3. import org.apache.iotdb.spark.db._
5. val df = spark.read.format("org.apache.iotdb.spark.db").option("url","jdbc:iotdb://127.0.0.1:6667/").option("sql","select * from root").
6. option("lowerBound", [lower bound of time that you want query(include)]).option("upperBound", [upper bound of time that you want query(include)]).
7. option("numPartition", [the partition number you want]).load
9. df.printSchema()
11. df.show()

3. 模式推理

以下 TsFile 结构为例： TsFile 架构中有三个度量：状态、温度和硬件。这三项测量的基本信息如下：

The existing data in the TsFile is as follows:

The wide(default) table form is as follows:

You can also use narrow table form which as follows: (You can see part 4 about how to use narrow form)

4. 宽表和窄表之间的转换

从宽到窄

1. import org.apache.iotdb.spark.db._
3. val wide_df = spark.read.format("org.apache.iotdb.spark.db").option("url", "jdbc:iotdb://127.0.0.1:6667/").option("sql", "select * from root where time < 1100 and time > 1000").load
4. val narrow_df = Transformer.toNarrowForm(spark, wide_df)

从窄到宽

1. import org.apache.iotdb.spark.db._
3. val wide_df = Transformer.toWideForm(spark, narrow_df)

5. Java 用户指南

1. import org.apache.spark.sql.Dataset;
2. import org.apache.spark.sql.Row;
3. import org.apache.spark.sql.SparkSession;
4. import org.apache.iotdb.spark.db.*
6. public class Example {
8. public static void main(String[] args) {
9. SparkSession spark = SparkSession
10. .builder()
11. .appName("Build a DataFrame from Scratch")
12. .master("local[*]")
13. .getOrCreate();
15. Dataset<Row> df = spark.read().format("org.apache.iotdb.spark.db")
16. .option("url","jdbc:iotdb://127.0.0.1:6667/")
17. .option("sql","select * from root").load();
19. df.printSchema();
21. df.show();
23. Dataset<Row> narrowTable = Transformer.toNarrowForm(spark, df)
24. narrowTable.show()
25. }
26. }