Partner/bondingadmin-install
12
0
Fork 0
Code Pull Requests Packages Projects Releases Activity

New migration tool DEV-3286 #1

Merged
alan merged 5 commits from DEV-3286-improve-management-server-migration-script into master 2021-10-01 18:08:16 -03:00
Conversation 12 Commits 5 Files Changed 2 +722 -19
jfunk commented 2021-09-15 10:26:10 -03:00
Owner
Copy Link
No description provided.
jfunk added 1 commit 2021-09-15 10:26:10 -03:00
jfunk commented 2021-09-15 10:26:32 -03:00
Author
Owner
Copy Link

See also https://wiki.cwk.multapplied.net/en/technical-support/bondingadmin/migration which has the new instructions

See also https://wiki.cwk.multapplied.net/en/technical-support/bondingadmin/migration which has the new instructions
alan commented 2021-09-17 02:36:19 -03:00
Owner
Copy Link

Probably want to install wget as well on

apt-get install -y gnupg
wget -qO - http://debian.multapplied.net/debian/public.gpg.key | apt-key add -
Probably want to install wget as well on ``` apt-get install -y gnupg wget -qO - http://debian.multapplied.net/debian/public.gpg.key | apt-key add - ```
alan commented 2021-09-17 04:02:53 -03:00
Owner
Copy Link

This will probably be used internally only but it might be better to have an echo for each step of commands.
For example when running migrate source, I came across the error

root@ba:/usr/src/bi/bondingadmin-install# ./migrate source

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

rm: missing operand
Try 'rm --help' for more information.

That was probably the apt install gnupg but I know I am definitely going to get calls from support because they have no idea if that was fine or not since there was no output on what was happening.

This will probably be used internally only but it might be better to have an echo for each step of commands. For example when running migrate source, I came across the error ``` root@ba:/usr/src/bi/bondingadmin-install# ./migrate source WARNING: apt does not have a stable CLI interface. Use with caution in scripts. rm: missing operand Try 'rm --help' for more information. ``` That was probably the apt install gnupg but I know I am definitely going to get calls from support because they have no idea if that was fine or not since there was no output on what was happening.
alan requested changes 2021-09-17 04:04:37 -03:00
migrate Outdated
@ -0,0 +310,4 @@
# Get config entries for the target install
#
for name in full_name short_name email country province city ; do
set_var $name "$(python3 -c \"import configparser; c=configparser.ConfigParser(); c.read('/etc/bondingadmin/bondingadmin.conf'); print(c['partner']['$name'])\")"
alan commented 2021-09-17 04:04:06 -03:00
Owner
Copy Link

Funny enough it worked if I used a freshly installed BA. But this error pops up when running source on my dev ba.

root@ba:/usr/src/bi/bondingadmin-install# ./migrate source

Pre-migration checks completed. To continue, run the following on the target
server in a tmux session:

    mkdir bondingadmin-install
    cd bondingadmin-install
    curl 'https://stash.multapplied.net/rest/api/latest/projects/PAR/repos/bondingadmin-install/archive?at=refs%2Fheads%2F6.6&format=tar.gz' | tar -xz
    ./migrate target-pre ba.comet.games

./migrate: command substitution: line 314: syntax error near unexpected token `('
Funny enough it worked if I used a freshly installed BA. But this error pops up when running source on my dev ba. ``` root@ba:/usr/src/bi/bondingadmin-install# ./migrate source Pre-migration checks completed. To continue, run the following on the target server in a tmux session: mkdir bondingadmin-install cd bondingadmin-install curl 'https://stash.multapplied.net/rest/api/latest/projects/PAR/repos/bondingadmin-install/archive?at=refs%2Fheads%2F6.6&format=tar.gz' | tar -xz ./migrate target-pre ba.comet.games ./migrate: command substitution: line 314: syntax error near unexpected token `(' ```
alan commented 2021-09-22 14:36:07 -03:00
Owner
Copy Link

Still not fixed.

set_var $name $(python3 -c "import configparser; c=configparser.ConfigParser(); c.read('/etc/bondingadmin/bondingadmin.conf'); print(c['partner']['$name'])") will fix it

Still not fixed. `set_var $name $(python3 -c "import configparser; c=configparser.ConfigParser(); c.read('/etc/bondingadmin/bondingadmin.conf'); print(c['partner']['$name'])")` will fix it
alan marked this conversation as resolved
alan added 1 commit 2021-09-17 04:07:16 -03:00
Merge branch 'master' into DEV-3286-improve-management-server-migration-script
Some checks failed
buildservice Build Debian_9.0/x86_64 result: broken Build Debian_8.0/x86_64 result: broken Build Debian_11/x86_64 result: broken Build Debian_10/x86_64 result: broken
959c68a8a1
jfunk commented 2021-09-20 11:29:44 -03:00
Author
Owner
Copy Link

This will probably be used internally only but it might be better to have an echo for each step of commands.
For example when running migrate source, I came across the error

root@ba:/usr/src/bi/bondingadmin-install# ./migrate source

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

rm: missing operand
Try 'rm --help' for more information.

That was probably the apt install gnupg but I know I am definitely going to get calls from support because they have no idea if that was fine or not since there was no output on what was happening.

> This will probably be used internally only but it might be better to have an echo for each step of commands. > For example when running migrate source, I came across the error > ``` > root@ba:/usr/src/bi/bondingadmin-install# ./migrate source > > WARNING: apt does not have a stable CLI interface. Use with caution in scripts. > > rm: missing operand > Try 'rm --help' for more information. > ``` > That was probably the apt install gnupg but I know I am definitely going to get calls from support because they have no idea if that was fine or not since there was no output on what was happening.
jfunk closed this pull request 2021-09-20 11:29:44 -03:00
jfunk commented 2021-09-20 11:30:41 -03:00
Author
Owner
Copy Link

This will probably be used internally only but it might be better to have an echo for each step of commands.

./migrate -x <whatever>

:-)

> This will probably be used internally only but it might be better to have an echo for each step of commands. ```bash ./migrate -x <whatever> ``` :-)
jfunk commented 2021-09-20 11:31:31 -03:00
Author
Owner
Copy Link

That was probably the apt install gnupg but I know I am definitely going to get calls from support because they have no idea if that was fine or not since there was no output on what was happening.

This is not for customers. It's for Goran.

> That was probably the apt install gnupg but I know I am definitely going to get calls from support because they have no idea if that was fine or not since there was no output on what was happening. This is not for customers. It's for Goran.
jfunk reopened this pull request 2021-09-22 13:30:40 -03:00
alan commented 2021-09-22 14:37:22 -03:00
Owner
Copy Link

Still need to fix that query command on line 314

Still need to fix that query command on line 314
jfunk commented 2021-09-23 07:12:29 -03:00
Author
Owner
Copy Link

Still need to fix that query command on line 314

LOL. Seems it didn't actually push my changes. Apparently there was an upstream change

> Still need to fix that query command on line 314 LOL. Seems it didn't actually push my changes. Apparently there was an upstream change
jfunk added 1 commit 2021-09-23 07:12:40 -03:00
Fix bad config option retrieval when the result has whitespace. Other
Some checks failed
buildservice Build Debian_9.0/x86_64 result: broken Build Debian_8.0/x86_64 result: broken Build Debian_11/x86_64 result: broken Build Debian_10/x86_64 result: broken
bb49f9120d 
minor fixes
jfunk requested review from alan 2021-09-23 07:12:52 -03:00
alan requested changes 2021-09-23 14:36:00 -03:00
migrate Outdated
@ -0,0 +305,4 @@
echo
echo " mkdir bondingadmin-install"
echo " cd bondingadmin-install"
echo " curl 'https://stash.multapplied.net/rest/api/latest/projects/PAR/repos/bondingadmin-install/archive?at=refs%2Fheads%2F$release&format=tar.gz' | tar -xz"
alan commented 2021-09-23 14:35:54 -03:00
Owner
Copy Link

We don't have stash anymore

We don't have stash anymore
alan marked this conversation as resolved
jfunk added 1 commit 2021-09-24 07:30:33 -03:00
Update tarball link
Some checks failed
buildservice Build Debian_9.0/x86_64 result: broken Build Debian_8.0/x86_64 result: broken Build Debian_11/x86_64 result: broken Build Debian_10/x86_64 result: broken
c37af1c121
alan requested changes 2021-09-25 21:19:09 -03:00
migrate
@ -0,0 +304,4 @@
echo "server in a tmux session:"
echo
echo " mkdir bondingadmin-install"
echo " cd bondingadmin-install"
alan commented 2021-09-24 14:37:13 -03:00
Owner
Copy Link
root@test:~# curl 'https://git.multapplied.net/Partner/bondingadmin-install/archive/master.tar.gz' | tar -xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1892  100  1892    0     0   142k      0 --:--:-- --:--:-- --:--:--  153k
root@test:~# ls
bondingadmin-install
root@test:~# ls -la
total 12
drwx------ 1 root root  122 Sep 24 10:34 .
drwxr-xr-x 1 root root  316 Jul  7 04:12 ..
-rw------- 1 root root 1989 Sep 24 10:34 .bash_history
-rw-r--r-- 1 root root  570 Jan 31  2010 .bashrc
drwxrwxr-x 1 root root   20 Sep  8 15:16 bondingadmin-install
drwxr-xr-x 1 root root   20 Sep 16 22:47 .config
drwxr-xr-x 1 root root   10 Sep 16 22:48 .local
-rw-r--r-- 1 root root  148 Aug 17  2015 .profile

The tar file unzips to create a bondingadmin-install folder. So the cd bondingadmin-install should be moved to after the curl

``` root@test:~# curl 'https://git.multapplied.net/Partner/bondingadmin-install/archive/master.tar.gz' | tar -xz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1892 100 1892 0 0 142k 0 --:--:-- --:--:-- --:--:-- 153k root@test:~# ls bondingadmin-install root@test:~# ls -la total 12 drwx------ 1 root root 122 Sep 24 10:34 . drwxr-xr-x 1 root root 316 Jul 7 04:12 .. -rw------- 1 root root 1989 Sep 24 10:34 .bash_history -rw-r--r-- 1 root root 570 Jan 31 2010 .bashrc drwxrwxr-x 1 root root 20 Sep 8 15:16 bondingadmin-install drwxr-xr-x 1 root root 20 Sep 16 22:47 .config drwxr-xr-x 1 root root 10 Sep 16 22:48 .local -rw-r--r-- 1 root root 148 Aug 17 2015 .profile ``` The tar file unzips to create a bondingadmin-install folder. So the *cd bondingadmin-install* should be moved to after the *curl*
alan marked this conversation as resolved
migrate
@ -0,0 +344,4 @@
# Check that locale matches the source
#
locale=$(su postgres -c "psql -Atc \"select datcollate from pg_catalog.pg_database where datname = 'bondingadmin';\"")
alan commented 2021-09-25 21:19:03 -03:00
Owner
Copy Link

Looks like the source_locale was never actually set. On the source BA, I can see that the the locale is C.UTF-8.

root@ba:~# su postgres -c "psql -Atc \"select datcollate from pg_catalog.pg_database where datname = 'bondingadmin';\""
could not change directory to "/root": Permission denied
C.UTF-8

On the new server, after it tried to set the locale it just gives the error

root@ba:/usr/src/bi/bondingadmin-install# ./migrate target-pre ba.comet.games
psql: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

Looking at the source locale-gen file on the new server, it appears that source_locale was always empty so it just broke the creation.

Looks like the source_locale was never actually set. On the source BA, I can see that the the locale is C.UTF-8. ``` root@ba:~# su postgres -c "psql -Atc \"select datcollate from pg_catalog.pg_database where datname = 'bondingadmin';\"" could not change directory to "/root": Permission denied C.UTF-8 ``` On the new server, after it tried to set the locale it just gives the error ``` root@ba:/usr/src/bi/bondingadmin-install# ./migrate target-pre ba.comet.games psql: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? ``` Looking at the source locale-gen file on the new server, it appears that source_locale was always empty so it just broke the creation.
alan marked this conversation as resolved
jfunk added 1 commit 2021-09-27 09:00:47 -03:00
Make it obvious an error occurred in the source check. Workaround Debian
Some checks failed
buildservice Build Debian_9.0/x86_64 result: broken Build Debian_8.0/x86_64 result: broken Build Debian_11/x86_64 result: broken Build Debian_10/x86_64 result: broken
4f8bb0955e 
brain-damage
alan requested changes 2021-09-27 21:29:39 -03:00
alan left a comment
Owner
Copy Link

That one should be the last issue with this.

That one should be the last issue with this.
migrate
@ -0,0 +349,4 @@
# Check that locale matches the source
#
locale=$(su postgres -c "psql -Atc \"select datcollate from pg_catalog.pg_database where datname = 'bondingadmin';\"")
alan commented 2021-09-27 21:27:29 -03:00
Owner
Copy Link
psql: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

So if I install from a fresh buster install and let migrate run install.sh to install, postgresql service hasn't started yet. A simple systemctl restart fixed this issue.

``` psql: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? ``` So if I install from a fresh buster install and let migrate run install.sh to install, postgresql service hasn't started yet. A simple systemctl restart fixed this issue.
jfunk commented 2021-09-28 08:08:27 -03:00
Author
Owner
Copy Link

That should have been started by bondingadmin-setup. If not, a bunch of stuff will be broken. Also, I am unable to replicate that behaviour here.

That should have been started by bondingadmin-setup. If not, a bunch of stuff will be broken. Also, I am unable to replicate that behaviour here.
alan merged commit 11e9c2919a into master 2021-10-01 18:08:16 -03:00
Sign in to join this conversation.
Reviewers
No Reviewers
alan
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
alan bijan foundry goran grace graeme jfunk jira josh maddie manthan philip
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Partner/bondingadmin-install#1
No description provided.
Delete Branch "DEV-3286-improve-management-server-migration-script"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?