Error Installing Oozie

[staylebait]

Discovered that Oozie wasn't installing when using this script on initilization, so fired up a cluster and manually ran the script only to get this error when running sudo apt-get install oozie:

sayle_matthews@cluster-1-m:~$ sudo apt-get install oozie
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  bigtop-tomcat oozie-client
The following NEW packages will be installed:
  bigtop-tomcat oozie oozie-client
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 306 MB of archives.
After this operation, 363 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://dataproc-bigtop-repo.storage.googleapis.com/v1.0/ dataproc/contrib bigtop-tomcat all 6.0.36-1 [5,376 k
B]
Get:2 http://dataproc-bigtop-repo.storage.googleapis.com/v1.0/ dataproc/contrib oozie-client all 4.2.0-1 [10.7 MB]
Get:3 http://dataproc-bigtop-repo.storage.googleapis.com/v1.0/ dataproc/contrib oozie all 4.2.0-1 [290 MB]        
Fetched 306 MB in 17s (17.6 MB/s)                                                                                 
Selecting previously unselected package bigtop-tomcat.
(Reading database ... 59410 files and directories currently installed.)
Preparing to unpack .../bigtop-tomcat_6.0.36-1_all.deb ...
Unpacking bigtop-tomcat (6.0.36-1) ...
Selecting previously unselected package oozie-client.
Preparing to unpack .../oozie-client_4.2.0-1_all.deb ...
Unpacking oozie-client (4.2.0-1) ...
Selecting previously unselected package oozie.
Preparing to unpack .../archives/oozie_4.2.0-1_all.deb ...
Unpacking oozie (4.2.0-1) ...
dpkg: error processing archive /var/cache/apt/archives/oozie_4.2.0-1_all.deb (--unpack):
 trying to overwrite '/usr/lib/oozie/lib/zookeeper-3.4.6.jar', which is also in package oozie-client 4.2.0-1
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Processing triggers for man-db (2.7.0.2-5) ...
Processing triggers for systemd (215-17+deb8u3) ...
Errors were encountered while processing:
 /var/cache/apt/archives/oozie_4.2.0-1_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Not sure how to get it working, but getting this on a clean cluster just running the script even.

[chimerasaurus]

When I try it, I get this.

sudo: /usr/lib/oozie/bin/ooziedb.sh: command not found
  ./oozie.sh: line 44:   <property>
    <name>hadoop.proxyuser.oozie.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.oozie.groups</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.${user.name}.hosts</name>
    <value>*</value>
  </property>
...

I do not have time to look @ this right at the moment but looks like something broke with an Oozie update.

[staylebait]

After a few more hours of working on this and investigating I have discovered it appears to be the Ubuntu apt-get repository hosted inside Google Cloud. Some update to Oozie must have broken it as you said which is what it appears to me as well.

With that said I was able to force an install to work by running the following commands:
sudo apt-get download oozie
sudo dpkg --force-all -i oozie_4.2.0-1_all.deb
sudo apt-get download oozie-client
sudo dpkg --force-all -i oozie-client_4.2.0-1_all.deb

After that I had to manually do some stuff with libext to get the UI running, but that appears to be normal for this so I am not including those commands.

[chimerasaurus]

@pmkc FYI

I am short on time so I cannot investigate this fully. Looks like a Bigtop issue or Oozie version issue.

[rkparkjr]

Any update on this? I am running into exactly the same issue described above.

[pmkc]

Sorry about that. Our packages got upstream updates from Apache Bigtop, and I'll update the initialization actions for the new bits.

[revanwabale]

I am getting exactly same issue. is this fixed ?

[manojpatil05]

Getting same problem. I had opened new issue but i am closing that and following this one.

[jgogstad]

Forcing installation with dpkg works (version is 4.3.0 now), but the install script runs into problems in the here document ending up in "bad substitution":

~$ gsutil cat gs://dataproc-guid-us/google-cloud-dataproc-metainfo/other-guid/oozie-test-m/dataproc-initialization-script-0_output
[snip]
+ cat
/etc/google-dataproc/startup-scripts/dataproc-initialization-script-0: line 51:   <property>
    <name>hadoop.proxyuser.oozie.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.oozie.groups</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.${user.name}.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.${user.name}.groups</name>
        <value>*</value>
  </property>
: bad substitution

Which is of course because bash sees the variable references in the here document and try to resolve them. Replacing this line with cat > core-site-patch.xml <<'EOF' fixes that. Now the cluster script runs fine, but the gcloud command fails with a generic master node failure and no logs:

gcloud dataproc clusters create oozie-test --initialization-actions gs://mybucket/oozie.sh
Waiting on operation [projects/myproject/regions/global/operations/GUID].
Waiting for cluster creation operation...done.
ERROR: (gcloud.dataproc.clusters.create) Operation [projects/myproject/regions/global/operations/GUID] failed: Failed to initialize node oozie-test-m.