FAQ and known issues¶
Cannot connect to PID server¶
Please ensure that the firewall is open for all PID hosts in your
[config:<project>] section, port 5671.
If you can’t connect to the server the publisher will produce the following errors:
$ esgpublish --test --project test --map test.test.map --service fileservice --noscan --thredds ... INFO 2017-11-14 18:50:23,729 Opening connection to RabbitMQ... INFO 2017-11-14 18:50:23,792 Connecting to 18.104.22.168:5671 with SSL ERROR 2017-11-14 18:50:24,042 Connection to 22.214.171.124:5671 failed: timeout WARNING 2017-11-14 18:50:24,042 Could not connect, 0 attempts left ERROR 2017-11-14 18:50:24,042 Could not connect to pcmdi10.llnl.gov/esgf-pid:5671: "Connection to 126.96.36.199:5671 failed: timeout" (connection failure after 0.313869 seconds) ...
Setting up PID credentials¶
In case the publisher fails with one of the following errors please make sure your
[config:<project>] section is set up correctly and does contain the
For more information see section The project config section
Section 'config:cmip6' not found in esg.ini
Option 'pid_credentials' missing in section [config:<project>] of esg.ini. Please contact your tier1 data node admin to get the proper values.
Publication to THREDDS fails¶
Missing permissions to write catalogs¶
If you don’t have the permissions to write to the THREDDS directory the publisher fails with:
$ esgpublish --project test --map test.test.map --service fileservice --noscan --thredds Traceback (most recent call last): ... File "/usr/local/uvcdat/2.2.0/lib/python2.7/site-packages/esgcet-3.1.0-py2.7.egg/esgcet/publish/publish.py", line 269, in publishDatasetList threddsOutput = open(threddsOutputPath, "w") IOError: [Errno 13] Permission denied: u'/esg/content/thredds/esgcet/1/test.test.v1.xml'
Make sure the (unix-) user you use for publication has write access to the THREDDS catalogs,
/esg/content/thredds/esgcet/. There are a number of ways to manage the permissions. You could, for instance change the group and group-permissions to a group the (unix-) user is member of.$ chgrp -R <group> /esg/content/thredds/esgcet/ $ chmod -R g+w /esg/content/thredds/esgcet/
tlsv1 alert unknown ca¶
If you run myproxy-logon as (unix-) user root it might fail with the following error:
$ myproxy-logon -s esgf-data.dkrz.de -l test_user -b -t 72 -o $HOME/.globus/certificate-file Error authenticating: GSS Major Status: Authentication Failed GSS Minor Status Error Chain: globus_gss_assist: Error during context initialization globus_gsi_gssapi: Unable to verify remote side's credentials globus_gsi_gssapi: SSLv3 handshake problems: Couldn't do ssl handshake OpenSSL Error: s3_pkt.c:1259: in library: SSL routines, function SSL3_READ_BYTES: tlsv1 alert unknown ca SSL alert number 48
It is not recommended to publish as (unix-) user root, please use another user for publication. The publication as root might also have a number of other side-effects.
Publication to Solr Index fails¶
Configuration file option missing¶
... esgcet.exceptions.ESGPublishError: Configuration file option missing: hessian_service_certs_location in section: DEFAULT, file=/esg/config/esgcet/esg.ini
Add the following to /esg/config/esgcet/esg.ini
hessian_service_certs_location = %(home)s/.globus/certificates
The certificates are fetched during the myproxy-login -b (bootstrap)
THREDDS does not unpublish¶
- Staring in v3.4.4 esgunpublish is reconfigured to use the REST API by default. You will need to unpublish with –skip-index and use a list of datasets with versions with the
.vNNformat instead of
Error Running Publisher in nohup (or other) scripted environment¶
- You may see an error like this following a stack trace with UVCDAT_ANONYMOUS_LOG:
... IOError: [Errno 9] Bad file descriptor
- You msut diasable the UVCDAT anonymous logging as the user prompt will repeat periodically. Add to your script before calling esgpublish.
$ export UVCDAT_ANONYMOUS_LOG=no
Version of this software¶
$ python -c "import esgcet ; print esgcet.__version__"
Recent versions may have a –version argument implemented and included in esgpublish.
$ esgpublish --version