With Sam's Backup and Restore script I need help

kayakbabe

I’ve read through this topic and several others about backing up magic mirror especially this one
Re: So I have a question about backup

I’m trying to use Sam’s backup and restore scripts.
I actually have it making files into a folder named MM_backup. But now I think my problem is that I don’t understand how git works all that well yet. I’ve only used it in a very simplistic way until now.

Should my repository on GitHub have the same name as my local MM_backup folder? I ask because what is happening is that when I try to connect my local folder to the repository I made on GitHub, I get error messages about a repository inside a repository.

I made a repository on GitHub then went to my local MM_backup and git cloned that repository into MM_backup.
Then I ran the backup script making sure I specified the correct repository name. Then i try to git add the new files. GitHub is having none of it. says i have a repository inside a repository.

SOOO. I thought aHA! i will change the backup folder path in the bash script. I deleted all the folders i made.
Created a folder at /home/reponame
Ran the script using the flag -b /home/reponame

but the script doesn’t put the files in /home/reponame. The script creates a folder MM_backup and puts the files it made in there.
So perhaps that is a bug?

I don’t understand what is happening.
And i can’t push the files the script made to the repository because i somehow made a repository inside a repository.

I manually added the files to my repository on Github in the hopes that they’ll be useful at some point.

I’d really like to make this work. But I need help.

sdetweil

@kayakbabe thanks for asking

we have 3 things here

the backup (and restore)

the scripts take parameters if you want to change from the defaults

-s is the MagicMirror folder location
-b is the backup folder location

the git repo

git creates a hidden folder (.git) to keep track of the content it is given to track.
you can make successive commits and retrieve/restore to any of those at any
time.
git provides a labeling mechanism,
so you don’t have to remember some gory details, called a tag.

the tag can be any label you like. I chose numbers. 1,2 3 .

the actual files are called the working copy

and you MUST also provide some
message (-m parm) when you make a commit to the repo, describing why you did that

so with the repo, you get
a saved collection of files
at different levels
ability to see what changed, a vs b
the ability to restore to any particular point at any time.
all in the same folder.

3 the github repo

BUT if the storage location (90% sd card) goes bad, you lose the repo too. oh no.
how to get the backup folder and repo somewhere ‘easily’
copy to another sd card, usb stick, another computer. all take figuring out new stuff.
hm. in MagicMirror we constantly use
git clone to get stuff from github.
and it’s cloud, so from anywhere to anywhere.

cool, github provides a way to copy a repo from local to cloud. using similar names, a
repository.
so you create a (private) repository on github (login id), and name. it can be any name you like.
now we have to connect the github repo, with the local repo
git provides a command
git remote add name url
name can be anything, but for command ease of use, it is typically origin
and there is a command to see the connection
git remote -v

the backup script saves that, for each module to know where to get it on restore

ok, now we have a link between here and github.

git also provides a quick way to change between collections of files
a branch, a fork in the road,
the default name is ‘main’

it is also a marker for files at some change point. we use this mechanism for MagicMirror updates. the main branch we update once every 90 days. new incoming updates go to the develop branch
and there is a tag added ever 90 days after the main update. so we/you can get to the collection of files at any release point.

now to get the stuff from your mirror system to github
git push (some parms I never can remember )

and that sends the hidden repo to github.
hurray!! safe and sound

hm. how do I get them back when I need them…
git provides a command for that
git clone repo_url
and voila, the files and repo are stored locally.

one quirk here, BY DEFAULT, the restored folder name will be the same as the github repo name UNLESS you provide an extra parm
git clone url folder_name
you an also rename the folder after git clone is finished.

but restore will do all that (but that points out a ‘bug’ in the script. maybe).

a couple more details
what good is a change marker (commit) if you don’t know WHO did it and how to reach them (git username and email)
I don’t ask for password, git will do that

the commands will dump.out help with
-help as the parm

ask more questions if I didn’t cover something

sdetweil

@kayakbabe ah, I see

-b xxxxx

ASSUMES in the $HOME folder
UNLESS the folder already exists

so -b xyz

means $HOME/xyz

but -b /foo/bar
means /foo/bar
ONLY IF /foo/bar already exists
otherwise I will jam it in
$HOME//foo/bar

note the double slash. maybe a bug

I didn’t want to require sudo to create a folder off the root (leading slash /)

note /home
is not the user home
that is $HOME

if you wanted reponame in your $HOME folder just do -b reponame

if you look at the log it might help too
~/MagicMirror//installers/backup.log

sdetweil

@kayakbabe I just added some more messages on the backup with a specified foldername…

(checked on double //)
/home/fred or /foo/bar
will create that in the $HOME folder $HOME/home/fred or $HOME/foo/bar

and on restore to the specified folder, if it exists but is NOT the repo folder abort

all go to the screen and logfile

and I checked on restore if the repo name doesn’t match the backup folder name then I add the extra parm on git clone if the folder doesn’t exist

if it exits, and IS a repo I git pull

if NOT a repo I exit before doing anything

sdetweil

@kayakbabe

and one other thing

only one side can create the repo with files

github (with readme), then git clone to pi (and add files via backup)
or
github create empty repo, no files
git push initial files from pi (after backup)

if u create repo at github w readme
AND you git commit at pi
then try to git pull,
now you have confusion
if you git clone, BUT it already exists
you can clone INSIDE the repo/backup
folder but now you have repo in repo.

kayakbabe

@sdetweil

I’ll put this in my own words to see if I’m understanding what I am to do.

Depending on how I create my repository on github will determine my next step.

Create respository on GitHub (private is most preferable since I might have private info in my magic mirror config file and I don’t want that public via GitHub).

Note: This process is easier if you create an empty GitHub Repository.

If I created an empty GitHub Repsitory:
2.a.Run the backupscript using the flags on the backup script for repository name, source folder, backup destination folder, github userid, etc.
2.b. from the backup desitnation folder run the command
git push
if I created GitHub repository with a readme:
then I’ve made a big mess and shouldn’t have done that. So delete the local folder and github repository and start over.

Results
I ran this

bash -c ./mm_backup.sh -s /home/pi/MagicMirror -b /home/pi/MMbackup2 -r https://github.com/Kayakbabe/MMbackup2.git -u Kayakbabe -e [email protected] -m 5thTry

And got this:
folder created with name /home/MM_backup
files created inside /home/MM_backup
no folder named MMbackup2

I also tried with -b /home/pi/MMbackup2
and still get home/pi/MM_backup created

i don’t have a folder named MMbackup2 on my pi at all.

I then manually created /home/pi/MMbackup2
ran my command again and still no files were placed in MMbackup2. i tried both =b /home/pi/MMbackup2 and -b MMbackup2 to no avail.

I noticed .git was created in the MM_backup folder along with the files the script created.

I ran git remote -v and get no output. (Just to make sure, i went to a diff local git folder and ran git remote -v and it shows the origins for fetch and pull.) so I think the .git being created in the MM_backup folder isn’t correct.

My questions are:

What is wrong with my flags that my backup folder isn’t being create with the folder name I chose?

Was the backup script supposed to do all the git stuff for me, like the git remote add origin and add files and set the branch and message?
So that, all I am supposed to have to run git push?

Or, am i supposed to now do all the following stuff?

git remote add origin https://github.com/<USER>/<repo>.git
git branch -M main
git push -u origin main

sdetweil

@kayakbabe can you send me the log?

MagicMirror/installers/backup.log

if you are putting the files in your home folder you don’t use the full path

you don’t push, -p will do that.
if you do NOT specify -p then YOU have to do the setup to push, and push. I was trying to save you work
(but as you specified the github repo, I should prompt if you DO want to push)

git remote -v
is from the inside backup folder (see below)

any parms whos default is good you don’t need to specify

also, you don’t clone the scripts to your system, just execute then from the guthub page like install and upgrade (you don’t get my fixes if you clone the repo),
you also don’t contaminate your system in case you have disk troubles and are trying to make an emergency backup

bash -c ./mm_backup.sh -s /home/pi/MagicMirror -b /home/pi/MMbackup2 -r https://github.com/Kayakbabe/MMbackup2.git -u Kayakbabe -e [email protected] -m 5thTry

should be (-s MagicMirror is default)

bash -c  "$(curl -sL https://raw.githubusercontent.com/sdetweil/MagicMirror-backup-restore/main/mm_backup.sh)" -p  -b MMbackup2 -r MMbackup2 -u Kayakbabe -e [email protected] -m "first backup after build finished (or whatever)"

because you did not specify to push, the script did not setup for push (no git remote add )

sdetweil

@kayakbabe
try the latest

I updated the scripts to check for and handle full paths like you used.
and if you specified a repo but did NOT request push, I prompt to discover if you WANT to push this time

if the full path of the backup folder does not exist, I will try to create it,
if it fails, I will report the error, and abort.

-b /foobar

will probably fail, with permission denied
-b /home/$username/whatever

will be used as specified, not jamming $HOME in front

thank you for these testcases which I hadn’t considered…

please use your prior full path command (but use from github)

I also extract the userid from the github repo (and if you specify it thru -u, I check that it matches the one in the url, and abort if not)
hm… I typically build the url from username abd reponame. one thing to check… thx

kayakbabe

@sdetweil
what was happening is that no matter if i used the full path or just the folder name like this
-b /home/pi/MMbackup2
-b MMbackup2
the script always used it’s own folder name of MM_backup.

when i tried to run from your repository,
Illegal option '- ’

I tried a local copy pulled a fresh copy from your GitHub and tried again
using -s MagicMirror
and -b MMbackup3

bash -c ./mm_backup.sh -p -b MMbackup3 -r MMbackup2 -u Kayakbabe -e [email protected] -m "second backup after build finished "

the backup folder was still created MM_backup and not MMbackup3.

Then I tried removing the -p option and I get this
bash -c “$(curl -sL https://raw.githubusercontent.com/sdetweil/MagicMirror-backup-restore/main/mm_backup.sh)” -b MMbackup2 -r MMbackup2 -u Kayakbabe -e [email protected] -m “first backup after build finished (or whatever)”
environment: line 155: [: too many arguments
checking for backup folder MMbackup2
folder doesn’t exist, creating backup folder /home/kelly/MMbackup2
backup folder is /home/kelly/MMbackup2
username specified with -u temp doesn’t match the user in the github repo MMbackup2, aborting

Note i used my correct email, just don’t want it out here to get scraped. And the name and email DO match my github.
I run git remote-v in the folder it created (MM_backup) and there is no git .

sdetweil

@kayakbabe well, all I can say, is I must not have tested it right.
my apologies

try again… all three scripts corrected now