MRS impala对接sentry，实现Impala权限控制
引言： impala 3.2.0版本通过对接sentry实现Impala的权限控制，本文介绍如何到MRS 服务大数据的impala（3.2.0）对接impala的操作，将sentry直接安装在MRS的master节点上。
安装Sentry Server
1. 编译安装Sentry Server
可使用开源sentry安装包https://mirrors.bfsu.edu.cn/apache/sentry/2.1.0/ （3.2版本impala对接2.1.0sentry即可），
备注：安装包也可以下载源码自己自己编译 //git clone https://github.com/cloudera/sentry.git
2. 设置环境变量：
解压Sentry压缩包到指定目录下，编辑/etc/profile， Sentry环境变量
3. 配置sentry-site.xml：
转到Sentry解压目录的conf文件夹下，修改sentry-site.xml配置文件：
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <configuration> <property> <name>sentry.service.server.rpc-address</name> <value>192.168.0.56</value> </property> <property> <name>sentry.service.server.rpc-port</name> <value>8038</value> </property> <property> <name>sentry.service.security.mode</name> <value>kerberos</value> </property> <property> <name>sentry.service.server.principal</name> <value>impala/node-master1bhjr.82f55c8f-71f7-431f-be97-fccc0495acc4.com@82F55C8F_71F7_431F_BE97_FCCC0495ACC4.COM</value> </property> <property> <name>sentry.service.server.keytab</name> <value>/opt/Bigdata/MRS_2.1.1/1_13_Catalog/etc/impala.keytab</value> </property> <property> <name>sentry.service.admin.group</name> <value>impala</value> </property> <property> <name>sentry.service.allow.connect</name> <value>qiyi_impala,impala</value> </property> <property> <name>sentry.store.group.mapping</name> <value>org.apache.sentry.provider.common.HadoopGroupMappingService</value> </property> <property> <name>sentry.service.reporting</name> <value>JMX</value> </property> <property> <name>sentry.service.web.enable</name> <value>true</value> </property> <property> <name>sentry.service.web.port</name> <value>51000</value> </property> <property> <name>sentry.service.web.authentication.type</name> <value>NONE</value> </property> <property> <name>sentry.verify.schema.version</name> <value>true</value> </property> <property> <name>sentry.store.jdbc.url</name> <value>jdbc:mysql://192.168.0.48:3306/sentry_test?useSSL=false</value> </property> <property> <name>sentry.store.jdbc.driver</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>sentry.store.jdbc.user</name> <value>root</value> </property> <property> <name>sentry.store.jdbc.password</name> <value>Bigdata_2019</value> </property> </configuration> |
sentry.service.server.rpc-address配置为安装sentry机器ip
sentry账号，替换以下sentry.service.server.principal，sentry.service.server.keytab，暂时服用了impala的服务kdc账号和keytab文件。
sentry.store.jdbc.url配置创建RDS（关系型数据库）url
1. 创建MySQL数据库表：
CREATE DATABASE `sentry_test` /*!40100 DEFAULT CHARACTER SET utf8 */;
2. 初始化Sentry数据库表：
将mysql-connector-java-5.1.47.jar放到Sentry解压目录的lib文件夹下，然后执行以下命令创建Sentry数据库表：
sentry --command schema-tool --conffile ${SENTRY_HOME}/conf/sentry-site.xml --dbType mysql --initSchema
3. 运行Sentry命令，启动Sentry服务端：
nohup sentry --command service --conffile ${SENTRY_HOME}/conf/sentry-site.xml>sentry.out 2>&1 &
4. 在浏览器输入以下地址访问Sentry Web UI，验证是否安装成功：
http://${ip}:51000/
Impala+Sentry整合
1、引入Sentry依赖
将apache-sentry-1.5.1-cdh5.16.1-bin/lib目录下相关jar拷贝到/opt/Bigdata/MRS_2.1.1/1_13_Catalog/install/impala/dependency目录下
2、创建sentry-site.xml
将apache-sentry-1.5.1-cdh5.16.1-bin/conf目录下的sentry-site.xml.service.template文件拷贝到所有节点的/etc/impala/conf（目录可自定义，与3中匹配即可，）目录下：
#
cp apache-sentry-1.5.1-cdh5.16.1-bin/conf/sentry-site.xml.service.template /etc/impala/conf/
# 重命名
cd /etc/impala/conf/
mv sentry-site.xml.service.template sentry-site.xml
编辑sentry-site.xml为以下内容：
<!--权限存储方式：数据库或者ini配置文件--> <property> <name>sentry.hive.provider.backend</name> <value>org.apache.sentry.provider.db.SimpleDBProviderBackend</value> </property> <!--权限认证方式，支持Kerberos认证，设置为none表示不启用认证 --> <property> <name>sentry.service.security.mode</name> <value>kerberos</value> </property> <property> <name>sentry.service.server.principal</name> <value>impala/node-master1bhjr.82f55c8f-71f7-431f-be97-fccc0495acc4.com@82F55C8F_71F7_431F_BE97_FCCC0495ACC4.COM</value> </property> <property> <name>sentry.service.server.keytab</name> <value>/opt/Bigdata/MRS_2.1.1/1_13_Catalog/etc/impala.keytab</value> </property> <property> <name>sentry.service.admin.group</name> <value>hive</value> </property> <property> <name>sentry.service.allow.connect</name> <value>qiyi_impala</value> </property> </configuration> |
sentry-site.xml 需要修改用户为omm，chown omm： sentry-site.xml；
3、启用权限认证
在服务端 服务Impala>服务配置>Impalad>自定义> impalad.gflagfile 中
增加--sentry_config=/etc/impala/conf/sentry-site.xml和--server_name=sentryserver配置
在服务端 服务Impala>服务配置>Catalog>自定义>catalogd.gflagfile 增加--sentry_config=/etc/impala/conf/sentry-site.xml
4、重启Impala服务，验证权限
重启Impala服务后，打开impala-shell，验证权限配置是否成功，具体操作如下：
（1）kinit qiyi_impala，打开impala-shell，创建一个admin角色：
[hadoop21-test1-rgtj5-tj1:21000] > create role admin_role;
Query: create role admin_role
Fetched 0 row(s) in 0.35s
（2）为admin角色赋予超级权限：
[hadoop21-test1-rgtj5-tj1:21000] > GRANT ALL ON SERVER sentryserver TO ROLE admin_role;
Query: GRANT ALL ON SERVER sentryserver TO ROLE admin_role
Query submitted at: 2019-07-06 10:40:11 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=15475b39691bd167:66c1403300000000
Fetched 0 row(s) in 0.13s
（3）将admin角色授权给impala用户组：
[hadoop21-test1-rgtj5-tj1:21000] > GRANT ROLE admin_role TO GROUP impala;
Query: GRANT ROLE admin_role TO GROUP hadoop
Query submitted at: 2019-07-06 10:41:53 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=434bb908587eaf31:65887a5a00000000
Fetched 0 row(s) in 0.11s
（4）创建一个test库和test表，并插入测试数据：
[hadoop21-test1-rgtj5-tj1:21000] > create database test;
Query: create database test
Fetched 0 row(s) in 0.29s
[hadoop21-test1-rgtj5-tj1:21000] > use test;
Query: use test
[hadoop21-test1-rgtj5-tj1:21000] > CREATE TABLE test(x INT, y STRING) STORED AS PARQUET;
Query: CREATE TABLE test(x INT, y STRING) STORED AS PARQUET
Fetched 0 row(s) in 0.16s
[hadoop21-test1-rgtj5-tj1:21000] > INSERT INTO test VALUES (1, 'one'), (2, 'two'), (3, 'three');
Query: INSERT INTO test VALUES (1, 'one'), (2, 'two'), (3, 'three')
Query submitted at: 2019-07-06 11:18:33 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=ce4e7f66f1209531:641f39a900000000
Modified 3 row(s) in 5.47s
因为impala用户是超级管理员并拥有ALL的权限，因此执行以下SELECT语句便能很快看到我们刚插入的数据：
[hadoop21-test1-rgtj5-tj1:21000] > select * from test;
Query: select * from test
Query submitted at: 2019-07-06 11:19:50 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=34e4b5594e3d0c6:8cfb1acb00000000
+---+-------+
| x | y |
+---+-------+
| 1 | one |
| 2 | two |
| 3 | three |
+---+-------+
Fetched 3 row(s) in 1.87s
(5) 接着我们切换到其他用户，运行impala-shell，对我们刚刚创建的test库进行操作：
[hadoop21-test1-rgtj5-tj1:21000] > use test;
Query: use test
ERROR: AuthorizationException: User 'root' does not have privileges to access: test.*.*
提示该用户没有操作test库的权限，至此，说明Sentry权限认证已经生效。
- 点赞
- 收藏
- 关注作者
评论（0）