Web lists-archives.com

Re: [Samba] AD Backup Best Practice




Hi Tim,

Thanks for taking the time to write this thorough answer.

If I understood you correctly, you're thinking of implementing the following improvements:

- Allow reuse of original DC name by removing the old DC from the database
- Automated move of current default Samba folder and files (which in all likelihood are anyway of no use) so that restore can be done in default folder and started using the regular unit file (systemd).

This clearly would make things a lot simpler.

I'd be happy to help test this, feel free to contact me directly.

Regards,
Viktor

On 11.02.2019 01:57, Tim Beale via samba wrote:
Hi,

Thanks for the feedback. I appreciate the whole backup/restore process
is more complicated than it needs to be (especially when you're trying
to restore in a panic).

I've explained below the technical reasons why the backup/restore has
these current limitations. The short answer is:
- We spent a huge amount of time implementing the current backup/restore
tool. Unfortunately, we just ran out of time to make the backup nicer to
use. We figured further improvements could be made in future, as needed.
- We're primarily samba developers, rather than samba users, so there's
some guesses we made as to how it'd be used/what's acceptable.

I think updating the restore tool to avoid these annoyances is not that
hard. Part of the hard part is just testing it manually to make sure we
get things right. On that note, would any samba users out there be
willing to volunteer to help test this? i.e. clone a development git
branch, build the source, and try out the backup/restore tool?

you *must not* specify a DC that has previously existed
The restore process currently works like this:
1. The backed up database (which still contains all the old DCs) is
restored to disk.
2. The new DC is added to the database.
3. All the previous DCs are removed from the database.

Note that:
- #3 has to happen to ensure the restored domain doesn't try to
communicate with the old/failed domain (just in case you forgot to
switch off a DC).
- The DC added in #2 will always be a "new" DC (even if it has the same
hostname, it will have different a invocation-ID, etc).
- You can't have zero DCs in the database, i.e. the new DC needs to be
added before all the old ones are removed.

Currently you can't add a new DC with the same name because it already
exists in the database. Yes, we could avoid this problem by checking the
database, and removing the old DC first. It was just more work than we
had time to implement.

restore the domain database into a different targetdir,
and then use the '-s' option when running samba
A domain join allows you to overwrite the existing samba database, which
is the behaviour you're asking for here. We wanted to avoid that in the
restore case because:
- If the restore didn't work successfully, you've just obliterated your
database.
- We want to be really careful that remnants from the old database
didn't get left behind, so wanted the restore to always be done into an
empty directory.

The simple solution would be to rename /usr/local/samba/ to
/usr/local/samba-old/ first, then run the restore. The problem here is
the samba-tool program that you'll use to do the restore is also in the
directory you just renamed.

Probably a better solution would be for the restore tool to move the
sub-directories it's going to restore first. i.e. automatically move
/usr/local/samba/private to /usr/local/samba/private-old.

Are you saying that restoring to a "new DC" is as simple as changing the
hostname of "DC1" to "DC1_1"? DNS, GPO, smb.conf would all
automatically refer to the new hostname after the restore?
So, a restore should be similar in functionality to a 'domain join',
i.e. the hostname specified in the command should get setup in smb.conf
and DNS.

Most of the sysvol/GPO backup/restore was implemented by another
developer, so I don't know that much about it. Basically all the sysvol
files and their NTACLs should be preserved as part of the backup
process. So you shouldn't need to do anything to the restored DC. As you
rejoin the DCs after the restore, you'd need to rsync the sysvol
directory (same as you would if you joined a new DC).

Let's assume the DC is in a filesystem that allows snapshots, do I
assume correctly that stopping samba would not be required in that case?
If you're foregoing the backup tools and just taking backing up the raw
files on disk, then you'd need to stop samba. Otherwise, I think you're
at risk of corruption from not holding appropriate DB locks.

The offline backup (in 4.10) is essentially just taking a snapshot of
the files on disk. However, it does it in a way that's safe WRT DB
locks. Samba doesn't need to be stopped in this case.

Also note that part of the point of the online backup is to try to
protect you from latent/benign database corruptions that might exist on
disk. i.e. the raw database on disk has a corruption but samba is still
running OK, but then that corrupt object gets modified again and
everything dies. If your backup has the same latent corruption, then
restoring it won't be much help long-term.

Regards,
Tim

On 11/02/19 5:40 AM, Rowland Penny via samba wrote:
I am cc'ing Tim Beale on this, he wrote the wikipage and, I think, most
of the backup & restore code.

On Sun, 10 Feb 2019 17:07:55 +0100
Viktor Trojanovic <viktor@xxxxxxxx> wrote:

See comments inline.

On Sun, 10 Feb 2019 at 16:33, Rowland Penny via samba
<samba@xxxxxxxxxxxxxxx> wrote:

On Sun, 10 Feb 2019 14:13:27 +0100
Viktor Trojanovic via samba <samba@xxxxxxxxxxxxxxx> wrote:

I'm currently reviewing my own backup strategy for Samba and I
realize it is not in line with best practices provided in the
Wiki.
( https://wiki.samba.org/index.php/Back_up_and_Restoring_a_Samba_AD_DC)
Said best practices, however, seem a bit like a nightmare to me.

Assuming the AD is gone and you want to restore just one DC, and
you want things to look just as they did before the crash, the
process according to the Wiki looks as follows:

1. Install a Samba DC on a new (!) temporary host and provision
the domain, just like you would when doing a new install from
scratch. That task alone is tremendous.
2. Stop Samba and restore the AD from backup to this domain not
(!) into the default Samba folder, advise Samba accordingly when
starting it. 3. On the original host, set up a Samba DC and join
the domain. 4. If GPO or scripts exist on sysvol, manually set up
sysvol replication to get them to the original DC.
5. Remove the temporary host.

Just... wow. :)
Tend to agree with you, the wiki page asks this question 'So which
backup should I use?' It then goes on to enumerate 5 different
reasons why you would need a backup and seems to totally miss the
point. Your domain has gone down and it is headless chicken time ;-)
All you would want to do is to get your domain back up again as
quickly as possible.

Yes. So I'm really glad I haven't encountered that page in a moment
of true need! :-)


I think you would only do '1' if you wanted to rename the domain.

Not sure where you got restoring into a different folder from, I
thought the restore put everything back to where it came from.


I got both of this from the section "Restoring the backup-file" in the
wiki. It says that if you're going to use the command "samba-tool
domain backup restore", you *must not* specify a DC that has
previously existed. Further, with regards to the files itself, it
says that the Samba team recommends "that you restore the domain
database into a different targetdir, and then use the '-s' option
when running samba".
OK, re-read the wiki-page and I do not understand why you have to do
this ?

Tim, Why do you have to do this ? I would image that, unless the DC
is kaput, Samba will be the only thing that needs fixing. Most
sysadmins would just clear out the Samba dirs and then carry out the
restore. I can understand renaming the DC, but why does it have to be
a new install ?
You shouldn't have to do '4', the backup contains a copy of sysvol
and smb.conf, so you should be able to restore to the DC it came
from, it would just have to be the only DC and all DC's would have
to be stopped, it would probably be better to rename the old DC
before carrying out the restore.


As mentioned, at least to my understanding of the the wiki a restore
of the original DC is not possible using the backup made from it. Are
you saying that restoring to a "new DC" is as simple as changing the
hostname of "DC1" to "DC1_1"? DNS, GPO, smb.conf would all
automatically refer to the new hostname after the restore?
That is my understanding and if not, Tim should be able to explain why
not ;-)

Isn't there a simpler way of doing this? Namely, if all the
restore operations are done offline anyway, why is it frowned
upon to simply do everything on the original DC, i.e. forgo the
temporary host, overwrite the configuration files (/etc/samba)
and the local Samba folder (e.g. /var/lib/samba) with what's in
the backup and be done with it? What's the difference between
doing this and just restoring the whole machine running the DC
bit for bit (dd backup and restore)?
If you are talking about stopping the DC and copying it (somehow),
then this should work, but you would have to be aware that you
would have to stop your DC regularly and that your backup would
only be valid for the time you took it, anything between that
backup and the next would be lost.

Let's assume the DC is in a filesystem that allows snapshots, do I
assume correctly that stopping samba would not be required in that
case?
The problem is that a Samba AD DC is constantly in flux, that is, it
changes constantly, if your 'snapshot' can guarantee it is correct,
then I see no problem, but you would only really know when you tried
to restore it.

With regards to information between 2 backups being lost, how
is that different with other backup strategies, for example using
samba-tool online backup?
That is the problem with any AD DC backup method, the backups can
quickly become out of date.

Rowland




--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba