Questions & Answers

pnda 3.5 salt orchestrate fails

0 votes
asked Aug 21, 2017 by Saqib Arfeen (560 points)
Hello guys
I am facing a problem while deploying applications on pnda.I am using local openstack cloud.Here is the output of salt-orchestrate command that runs as part of pnda deployment process.
please provide help/hints!
Thanks

cation2108-saltmaster_master:
----------
          ID: cdh-run_cloudera_user
    Function: salt.state
      Result: True
     Comment: States ran successfully.
     Started: 13:32:28.327460
    Duration: 623.763 ms
     Changes:
----------
          ID: cdh-install_cloudera_manager
    Function: salt.state
      Result: True
     Comment: States ran successfully.
     Started: 13:32:28.955635
    Duration: 540.303 ms
     Changes:
----------
          ID: cdh-install-agents
    Function: salt.state
      Result: True
     Comment: States ran successfully.
     Started: 13:32:29.499845
    Duration: 508.941 ms
     Changes:
----------
          ID: cdh-install_hadoop
    Function: salt.state
      Result: True
     Comment: States ran successfully.
     Started: 13:32:30.012948
    Duration: 529.585 ms
     Changes:
----------
          ID: cdh-create_master_dataset
    Function: salt.state
      Result: False
     Comment: Run failed on minions: cation2108-hadoop-edge
              Failures:
                  cation2108-hadoop-edge:
                      Data failed to compile:
                  ----------
                      Rendering SLS 'base:master-dataset' failed: Jinja error: HTTPConnectionPool(host='none', port=7180): Max retries exceeded with url: /api/v11/clusters/cation2108/services/hdfs01/nameservices (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe30465b050>: Failed to establish a new connection: [Errno -2] Name or service not known',))
                  Traceback (most recent call last):
                    File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 366, in render_jinja_tmpl
                      output = template.render(**decoded_context)
                    File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 969, in render
                      return self.environment.handle_exception(exc_info, True)
                    File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 742, in handle_exception
                      reraise(exc_type, exc_value, tb)
                    File "<template>", line 4, in top-level template code
                    File "/var/cache/salt/minion/extmods/modules/pnda.py", line 29, in namenodes_ips
                      named_service = get_named_service(cm_host, cm_name, service_name)
                    File "/var/cache/salt/minion/extmods/modules/pnda.py", line 10, in get_named_service
                      r = requests.get(request_url, auth=(user_name, password))
                    File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 72, in get
                      return request('get', url, params=params, **kwargs)
                    File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 58, in request
                      return session.request(method=method, url=url, **kwargs)
                    File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 508, in request
                      resp = self.send(prep, **send_kwargs)
                    File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 618, in send
                      r = adapter.send(request, **kwargs)
                    File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 508, in send
                      raise ConnectionError(e, request=request)
                  ConnectionError: HTTPConnectionPool(host='none', port=7180): Max retries exceeded with url: /api/v11/clusters/cation2108/services/hdfs01/nameservices (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe30465b050>: Failed to establish a new connection: [Errno -2] Name or service not known',))

                  ; line 4
                  ---
                  {% set pnda_user  = pillar['pnda']['user'] %}
                  {% set pnda_group = pillar['pnda']['group'] %}

                  {% set namenodes_ips = salt['pnda.namenodes_ips']() %}    <======================
                  # Only take the first one
                  {% set namenode = namenodes_ips[0] %}

                  {% set pnda_master_dataset_location = pillar['pnda']['master_dataset']['directory'] %}
                  {% set pnda_kite_dataset_uri = "dataset:hdfs://" + namenode + ":8020" + pnda_master_dataset_location %}
                  [...]
                  ---
     Started: 13:32:30.546452
    Duration: 12815.46 ms
     Changes:

Summary for cation2108-saltmaster_master
------------
Succeeded: 4
Failed:    1
------------
Total states run:     5

1 Answer

0 votes
answered Aug 28, 2017 by jeremie (3,730 points)

looks like you have a connectivity issue between edge and namenode. Once connected on the saltmaster, could you try to run:

sudo salt -v -C "G@pnda_cluster:cation and G@roles:cloudera_namenode" test.ping

and also:

sudo salt -v -G "roles:console_frontend" mine.get "roles:cloudera_namenode" network.ip_addrs expr_form=grain

commented Sep 15, 2017 by John O'Loughlin (160 points)
reshown Sep 15, 2017 by John O'Loughlin
"Comment Removed"
commented Nov 8, 2017 by divyas7 (140 points)
Hi,

I'm trying to install pnda on AWS . But got the same error :

 ID: cdh-create_master_dataset
2017-11-07 09:06:28 - INFO - 10.0.1.26     Function: salt.state
2017-11-07 09:06:28 - INFO - 10.0.1.26       Result: False
2017-11-07 09:06:28 - INFO - 10.0.1.26      Comment: Run failed on minions: testing-hadoop-edge
2017-11-07 09:06:28 - INFO - 10.0.1.26               Failures:
2017-11-07 09:06:28 - INFO - 10.0.1.26                   testing-hadoop-edge:
2017-11-07 09:06:28 - INFO - 10.0.1.26                       Data failed to compile:
2017-11-07 09:06:28 - INFO - 10.0.1.26                   ----------
2017-11-07 09:06:28 - INFO - 10.0.1.26                       Rendering SLS 'base:master-dataset' failed: Jinja error: HTTPConnectionPool(host='None', port=7180): Max retries exceeded with url: /api/v11/clusters/testing/services/hdfs01/nameservices (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)

let me know what can be done
commented Nov 8, 2017 by Saqib Arfeen (560 points)
Hello Divyas
My understanding is that you are running everything on master branch. I would suggest to post output of #git branch in "pnda-heat-templates" and "pnda" (cloned repositories). If it is on *master, then please run this command inside each cloned project.(pnda-heat-templates,pnda etc )
#git checkout release/3.5
Now please repeat the process all over like:-
./heat_cli -e cation -f pico -s cation -n 3
...