如何使用java调用数据迁移服务（CDM）的rest-api

CDM（数据迁移服务）是华为云提供的界面向导式数据迁移服务，支持对关系数据库、NoSQL数据库、大数据、搜索、FTP等20多种数据源的导入导出。与传统数据迁移工具使用INSERT INTO语句导入不同，CDM会利用MYSQL本地API加速数据导入，导入速度大约提升10倍。

本文演示如何使用Java调用云数据迁移服务REST API创建、启动、查询、删除CDM作业。调用这些接口，需要提前准备以下工作：

1、准备登录华为云的账号名、用户名和密码。

2、创建一个云数据迁移集群

3、获取项目ID：在控制台上打开云数据迁移,然后点开网页右上角的用户名的下拉菜单项“我的凭证”，“中国华北区1”的项目ID就是本用例使用的项目ID。类似：1551c7f6c808414d8e9f3c514a170f2e

4、获取集群ID：点开云数据迁移集群详情，拷贝浏览器地址栏上instanceInfo/后的一串字符就是集群ID，类似：c110beff-0f11-4e75-8b10-da7cd882b0ef

5、准备好依赖jar: httpclient-4.5.5.jar

目 录

一、使用java调用登录接口，并获取登录token。

二、创建数据迁移作业接口

三、查询作业运行状态接口

四、删除作业接口

一、使用java调用登录接口，并获取登录token。

登录和获取token是进行其他相关操作的前提。要顺利的获取token,首先需要准备以下：

1.填写request-json:

{auth:{"identity":{"methods":["password"],"password":{"user":{"name":"USER_NAME","password":"PASSWORD","domain":{"name":"DOMAIN_NAME"}}}},"scope":{"project": {"name": "cn-north-1"}}}}

USER_NAME：华为云的登录用户，在控制台上打开云数据迁移,然后点开网页右上角的用户名的下拉菜单项“我的凭证”，里面有用户名。

PASSWORD：华为云的登录密码

DOMAIN_NAME：华为云登录账号名，在控制台上打开云数据迁移,然后点开网页右上角的用户名的下拉菜单项“我的凭证”，里面有帐户名。

2. 调用获取token Api的地址： 以华北1为例，https://iam.cn-north-1.myhuaweicloud.com /v3/auth/tokens。改地址在其他区域（iam.cn-north-1部分）会有所变化。

确认好以上信息后，调用获取token的rest-api，示例代码如下：

 CloseableHttpClient httpclient =  HttpClients.createDefault();
 HttpPost getTokenPost = new HttpPost("https://iam.cn-north-1.myhuaweicloud.com /v3/auth/tokens ");
 String getTokenJson = "{\"auth\":{\"identity\":{\"methods\":[\"password\"],\"password\":{\"user\":{\"name\":\""
     + USER_NAME + "\",\"password\":\"" + PASSWORD + "\",\"domain\":{\"name\":\"" + DOMAIN_NAME + "\"}}}},\"scope\":{\"project\": {\"name\": \"cn-north-1\"}}}}";
 StringEntity tokenEntity = new StringEntity(getTokenJson);
 tokenEntity.setContentEncoding("UTF-8");
 tokenEntity.setContentType("application/json");
 getTokenPost.setEntity(tokenEntity);
 CloseableHttpResponse response = httpclient.execute(getTokenPost);
 Header tokenHeader = response.getFirstHeader("X-Subject-Token");
 token = tokenHeader.getValue();

二、创建数据迁移作业接口

调用数据迁移作业接口需要做以下准备：

1.调用创建数据迁移服务接口地址: https://cdm.cn-north-1.myhwclouds.com /cdm/rest/cdm/v1.0/PROJECT_ID}/cluster/{CLUSTER_ID /cdmui/sqoop/v1/job/，其中PROJECT_ID为项目ID，CLUSTER_ID为集群ID，两者的获取方法在本文开始有详细介绍。

2.token：按照本文第一节，获取到的登录口令

3.请求body-json，请求体非常复杂，且每个版本有一定的变化，可以先在作业管理界面上创建作业，然后点开此作业的操作菜单项“作业JSON定义”，然后复制JSON内容，格式化为Java字符串语法，然后粘贴到代码中：

{"jobs":[{"job_type":"NORMAL_JOB","to-config-values":{"configs":[{"inputs":[{"name":"toJobConfig.schemaName","value":"deng"},{"name":"toJobConfig.tablePreparation","value":"DROP_AND_CREATE"},{"name":"toJobConfig.tableName","value":"A0315"},{"name":"toJobConfig.columnList","value":"AA&BB&CC&DD"},{"name":"toJobConfig.isCompress","value":"false"},{"name":"toJobConfig.orientation","value":"ROW"},{"name":"toJobConfig.useStageTable","value":"false"},{"name":"toJobConfig.shouldClearTable","value":"false"},{"name":"toJobConfig.extendCharLength","value":"false"}],"name":"toJobConfig"}]},"from-config-values":{"configs":[{"inputs":[{"name":"fromJobConfig.schemaName","value":"sqoop"},{"name":"fromJobConfig.tableName","value":"test1"},{"name":"fromJobConfig.columnList","value":"AA&BB&CC&DD"},                          {"name":"fromJobConfig.incrMigration","value":"true"}],"name":"fromJobConfig"}]},"to-connector-name":"generic-jdbc-connector","from-link-name":" TOLINKNAME ","from-connector-name":"generic-jdbc-connector","to-link-name":" TOLINKNAME" ,"driver-config-values":{"configs":[{"inputs":[{"name":"throttlingConfig.numExtractors","value":"1"},{"name":"throttlingConfig.submitToCluster","value":"false"},{"name":"throttlingConfig.numLoaders","value":"1"},{"name":"throttlingConfig.recordDirtyData","value":"false"}],"name":"throttlingConfig"},{"inputs":[],"name":"jarConfig"},{"inputs":[],"name":"schedulerConfig"},{"inputs":[],"name":"transformConfig"},{"inputs":[{"name":"smnConfig.agencies","value":"NONE"},{"name":"smnConfig.selected","value":"NONE"}],"name":"smnConfig"},{"inputs":[],"name":"retryJobConfig"}]},"name":"JOBNAME"}]}

其中：TOLINKNAME为目标连接名称，FROMLINKNAME为源连接名称，JOBNAME为迁移作业名称。

调用示例如下：

 HttpPost httpPost = new HttpPost("https cdm.cn-north-1.myhwclouds.com /cdm/rest/cdm/v1.0/PROJECT_ID}/cluster/{CLUSTER_ID/cdmui/sqoop/v1/job/")
 String json = body-json;
 StringEntity s = new StringEntity(json);
 s.setContentEncoding("UTF-8");
 s.setContentType("application/json");
 httpPost.setEntity(s);
 httpPost.addHeader("X-Auth-Token", token);
 httpPost.addHeader("X-Language", "zh-cn");
 CloseableHttpResponse response = httpclient.execute(httpPost);
 int status = response.getStatusLine().getStatusCode();

三、查询作业运行状态接口

1.     接口地址: https://cdm.cn-north-1.myhwclouds.com /cdm/rest/cdm/v1.0/PROJECT_ID}/cluster/{CLUSTER_ID /cdmui/sqoop/v1/job/{JOB_NAME}/status, 其中PROJECT_ID为项目ID，CLUSTER_ID为集群ID，JOB_NAME为作业名称。

2.     token

调用示例如下：

       HttpGet httpGet = new HttpGet("https://cdm.cn-north-1.myhwclouds.com /cdm/rest/cdm/v1.0/PROJECT_ID}/cluster/{CLUSTER_ID /cdmui/sqoop/v1/job/{JOB_NAME}/status
        CallCdmApi callCdmApi = new CallCdmApi();
        CloseableHttpClient httpclient =  HttpClients.createDefault();
//      String token = callCdmApi.getToken();
        httpGet.addHeader("X-Auth-Token", token);
        httpGet.addHeader("X-Language", "zh-cn");
         CloseableHttpResponse response = httpclient.execute(httpGet);
         int status = response.getStatusLine().getStatusCode();

四、删除作业接口

1.     接口地址：https://cdm.cn-north-1.myhwclouds.com /cdm/rest/cdm/v1.0/PROJECT_ID}/cluster/{CLUSTER_ID /cdmui/sqoop/v1/job/{JOB_NAME}/

2.     token

调用示例如下：

 HttpDelete httpDelte = new HttpDelete("https://cdm.cn-north-1.myhwclouds.com/cdm/rest/cdm/v1.0/PROJECT_ID}/cluster/{CLUSTER_ID /cdmui/sqoop/v1/job/{JOB_NAME}/");
 //  httpDelte.setConfig(this.requestConfig);
 httpDelte.addHeader("X-Auth-Token", this.token);
 httpDelte.addHeader("X-Language", "zh-cn");
 CloseableHttpResponse response = httpclient.execute(httpDelte);
int status = response.getStatusLine().getStatusCode();