Configuring External Kerberos Server for HA EMR

EMR with multiple master nodes requires external Kerberos.

Following is the procedure to configure the Kerberos server:

Launching EC2 Server

  • Login to AWS console and navigate to EC2.

  • Click the Launch Instance button.


  • In the Choose an Amazon Machine Image (AMI) page, click the Select button for the Amazon Linux 2 AMI.


  • In the next steps, select the VPC, security group, and key for authentication and launch an instance. This is the External Kerberos server.

NOTE Ensure that the 749 and 88 ports are open to the EMR cluster and Edge node.

Configuring Network and Keys to SSH

  • SSH into the Kerberos server.

  • Install the Kerberos server packages using the following command:

sudo yum install krb5-libs krb5-server krb5-workstation

  • Define the Kerberos realm for the cluster by modifying the /etc/krb5.conf configuration file. The following is an example to configure a Kerberos server with a realm, REALM.DOMAIN (INFOWORKS.IO), on a host, ip-10-00-03-218.ec2.internal.

[libdefaults]
default_realm = INFOWORKS.IO
dns_lookup_realm = false
dns_lookup_kdc = false
rdns = true
ticket_lifetime = 24h
forwardable = true
udp_preference_limit = 1000000
default_tkt_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 des3-cbc-sha1
default_tgs_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 des3-cbc-sha1
permitted_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 des3-cbc-sha1

[realms]

INFOWORKS.IO = {
kdc = ip-10.30.1.3.ec2.internal
admin_server = ip-10.30.1.3.ec2.internal
default_domain = ec2.internal
}

[domain_realm]
.ec2.internal = INFOWORKS.IO
ec2.internal = INFOWORKS.IO
[logging]
  • Define the Kerberos realm for the cluster by modifying the /var/kerberos/krb5kdc/kdc.conf configuration file. The following is an example:

default_realm = INFOWORKS.IO

[kdcdefaults]
# v4_mode = nopreauth
kdc_tcp_listen_backlog = 38000
kdc_ports = 0

[realms]
INFOWORKS.IO = {
kdc_ports = 88
admin_keytab = /etc/kadm5.keytab
database_name = /var/kerberos/krb5kdc/principal
acl_file = /var/kerberos/krb5kdc/kadm5.acl
key_stash_file = /var/kerberos/krb5kdc/stash
max_life = 10h 0m 0s
max_renewable_life = 7d 0h 0m 0s
master_key_type = des3-hmac-sha1
supported_enctypes = aes256-cts-hmac-sha1-96:normal aes128-cts-hmac-sha1-96:normal des3-cbc-sha1:normal
# supported_enctypes = des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal des-cbc-crc:v4 des-cbc-crc:afs3
# default_principal_flags = -preauth
  • If required, modify the /var/kerberos/krb5kdc/kadm5.acl file to grant the appropriate permissions to the admin.

  • Create the KDC database using the following command:

kdb5_util create -s

  • Enter the appropriate password for the KDC database master key.

  • Start the Kerberos daemons using the following commands:

root@kdc-server$ /sbin/service krb5kdc start

root@kdc-server$ /sbin/service kadmin start

  • To start Kerberos automatically on system restart, run the following commands:

root@kdc-server$ /sbin/chkconfig krb5kdc on

root@kdc-server$ /sbin/chkconfig kadmin on