2 Replies Latest reply: Sep 4, 2019 4:24 AM by Trey RSS

    Another failed upgrade from 1.5 to 1.6

    Trey

      The first dozen threads of this forum are all about failed upgrades of varying degrees from this upgrade.

       

      I am one of the ones getting a 503 error.  No matter how many times I go through this process:

       

      https://learningnetwork.cisco.com/docs/DOC-36658#jive_content_id_IMPORTANT__Start_Here

       

      The upgrade stops at this point

       

      Running (locally) common.arp.init...

      {

        "file_|-enlarge_arp_cache_|-/usr/local/bin/enlarge_arp_cache_|-managed": {

          "comment": "User virl-services is not available Group virl-services is not available",

          "pchanges": {},

          "name": "/usr/local/bin/enlarge_arp_cache",

          "start_time": "01:48:12.691691",

          "result": false,

          "duration": 5.538,

          "__run_num__": 0,

          "__sls__": "common.arp.install",

          "changes": {},

          "__id__": "enlarge_arp_cache"

        },

        "cmd_|-run_enlarge_arp_cache_|-sysctl -p_|-run": {

          "comment": "Command \"sysctl -p\" run",

          "name": "sysctl -p",

          "start_time": "01:48:12.706995",

          "result": true,

          "duration": 7.592,

          "__run_num__": 2,

          "__sls__": "common.arp.reload",

          "changes": {

            "pid": 2353,

            "retcode": 0,

            "stderr": "",

            "stdout": "net.ipv4.ip_forward = 1"

          },

          "__id__": "run_enlarge_arp_cache"

        },

        "cmd_|-append_config_|-/usr/local/bin/enlarge_arp_cache_|-run": {

          "comment": "onlyif execution failed",

          "name": "/usr/local/bin/enlarge_arp_cache",

          "start_time": "01:48:12.697878",

          "skip_watch": true,

          "__id__": "append_config",

          "duration": 8.816,

          "__run_num__": 1,

          "__sls__": "common.arp.install",

          "changes": {},

          "result": true

        }

      }

      === State common.arp.init ===

      Succeeded 2; failed 1

      Duration: 2.85

      ERROR: Some states failed in call "state.sls common.arp.init":

      "file_|-enlarge_arp_cache_|-/usr/local/bin/enlarge_arp_cache_|-managed" (User virl-services is not available Group virl-services is not available)

      Upgrade failed on common.arp.init

       

      So at this point, I would love to simply redeploy to 1.6, but I can't find the download page for the OVA, so I'm pretty screwed, I don't want to re-install what I have and run the risk of the licensing being totally hosed, since there's no real support for this product which costs actual money.

       

      I went through about a dozen failed upgrade threads, but couldn't find this particular failure.   Running the command advised in the document returns the same error I pasted above:

       

      virl@virl:~$ cd /var/local/virl/logs ; grep -l 'vinstall upgrade' *.cmd | while read ff ; do tail ${ff/cmd/out} ; done

          "changes": {},

          "result": true

        }

      }

      === State common.arp.init ===

      Succeeded 2; failed 1

      Duration: 2.95

      ERROR: Some states failed in call "state.sls common.arp.init":

      "file_|-enlarge_arp_cache_|-/usr/local/bin/enlarge_arp_cache_|-managed" (User virl-services is not available Group virl-services is not available)

      Upgrade failed on common.arp.init

        • 1. Re: Another failed upgrade from 1.5 to 1.6
          Karlo Bobiles

          Hello Trey,

           

          Looping in rrozar for further investigation. If you'd like to just redeploy version 1.6.65, please use the following link to download the latest image:

           

          https://learningnetworkstore.cisco.com/

           

          Click on "My Accounts" located towards the top right corner. Find your VIRL Purchase Order form the list, and select "Download"

           

          Just make sure to shutdown your current evironment prior to activating your license.

           

          Thank you,

          Karlo Bobiles

          Cisco Learning Network

          • 2. Re: Another failed upgrade from 1.5 to 1.6
            Trey

            If this is a recoverable error, then fine, let's recover it. I am downloading the 1.6 OVA just in case, but my installation was just fine until this upgrade attempt.  As with all Cisco products, getting the licensing to initially function properly was a frustrating, time consuming nightmare... Is this "state.sls common.arp.init" component related to licensing?  If so it's not intuitive...

             

            Apache appears to be running if I check the running processes and review the listening services via netstat, so it doesn't appear to be an issue with apache.

             

            Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.4.0-159-generic x86_64)

             

              System information as of Wed Sep  4 06:15:11 CDT 2019

             

              System load:     0.09               IP address for eth0:    192.168.1.2

              Usage of /:      30.2% of 59.38GB   IP address for br1:     172.16.1.250

              Memory usage:    35%                IP address for br2:     172.16.2.250

              Swap usage:      0%                 IP address for br3:     172.16.3.250

              Processes:       238                IP address for br4:     172.16.10.250

              Users logged in: 0                  IP address for docker0: 172.17.0.1

             

              Graph this data and manage this system at:

                https://landscape.canonical.com/

            Last login: Sun Sep  1 02:06:14 2019

            virl@virl:~$ ps -aux | grep apache

            root     11003  0.0  0.0  91232  5920 ?        Ss   05:55   0:00 /usr/sbin/apache2 -k start

            www-data 11008  0.0  0.1 448172 10332 ?        Sl   05:55   0:00 /usr/sbin/apache2 -k start

            www-data 11009  0.0  0.1 382564 10332 ?        Sl   05:55   0:00 /usr/sbin/apache2 -k start

            virl     19262  0.0  0.0  14224   980 pts/0    S+   06:15   0:00 grep --color=auto apache

             

            Specific processes identified via netstat -l

             

            unix  2      [ ACC ]     STREAM     LISTENING     45128    /var/run/docker/libnetwork/81aefce581dc8a4987abb46cdd6163e961187e8b173bb0ca23dd304de0bdbda7.sock

            unix  2      [ ACC ]     STREAM     LISTENING     42407    /var/run/salt/master/publish_pull.ipc

            unix  2      [ ACC ]     STREAM     LISTENING     42410    /var/run/salt/master/master_event_pub.ipc

            unix  2      [ ACC ]     STREAM     LISTENING     42411    /var/run/salt/master/master_event_pull.ipc

            unix  2      [ ACC ]     STREAM     LISTENING     42465    /var/run/salt/master/workers.ipc

            unix  2      [ ACC ]     STREAM     LISTENING     32121    /var/run/apache2/wsgi.11003.0.1.sock

            unix  2      [ ACC ]     STREAM     LISTENING     24821    /var/run/libvirt/virtlockd-sock

            unix  2      [ ACC ]     STREAM     LISTENING     24823    /var/run/pcscd/pcscd.comm

            unix  2      [ ACC ]     STREAM     LISTENING     24824    /run/uuidd/request

            unix  2      [ ACC ]     STREAM     LISTENING     24825    /var/run/dbus/system_bus_socket

            unix  2      [ ACC ]     STREAM     LISTENING     32122    /var/run/apache2/wsgi.11003.0.2.sock

             

            I did re-run the "docker" command from one of the other "failed upgrade" threads, but it ran with "7 successful, 0 changes"...

             

            It's taking a while to download this new OVA.