Web lists-archives.com

Re: Measuring (or calculating) how many bytes are actually written to disk when I repeatedly save a file




On 06.04.2019 22:39, rhkramer@xxxxxxxxx wrote:
Background: I am considering buying a new disk (and will write an email later 
with some other questions or observations about the process), but I know that, 
at least often for SSD drives, they now specify what I will call the longevity 
in terms of TB TBW (iiuc, that is terabytes total bytes written).

Anyway, I edit large files many times a day and try to save it at each edit or 
partial edit (at a guess, one particular file is around 100 MB, and I may save 
it 200 or more times a day).

There are two things I'd like to measure, and I'm wondering what tools (or 
approaches) are available:

1. I'd like to count how many times a day I actually save the file.  (One 
approach (at least I think I could do this) could be to write a sort of shell 
script wrapper and always initiate saves using the shell script, but I was 
hoping there was more of pre-built solution.)

2. A lot of my editing involves editing near (but not at) the end of a file.  I 
assume (I know) that the software that saves the file is smart enough not to 
rewrite the entire file but instead to preserve the beginning of the file and 
just rewrite the changed part of the file (or from there to the end of the 
file).

Can anyone confirm that, and, if so, suggest any way of measuring how much is 
written to a given file in a given time period (e.g., per day)?

I guess at a very deep level (I mean like at the level of the disk firmware or 
driver level), this may differ between an SSD and an HDD -- if you have any 
insight into that, I'd appreciate that.

Thanks!

As far as I understand, your final goal is to know how much data was written per day.
One way to find out is to use 'smartctl' utility and collect values daily from attribute #241 called "Lifetime_Writes_GiB".
Some SSD vendors could have it under different ID number or different name.

$ sudo smartctl -A /dev/sdb
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.15.0-0.bpo.2-amd64] (local build)
...
  9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       13244
 12 Power_Cycle_Count       0x0012   100   100   000    Old_age   Always       -       1350
...
241 Lifetime_Writes_GiB     0x0012   100   100   000    Old_age   Always       -       3526
242 Lifetime_Reads_GiB      0x0012   100   100   000    Old_age   Always       -       4581
...

Of course, if you are not familiar with programming, or not interested in writing a text parser, to setup a cronjob and
possibly to install monitoring software with fancy graphs, then my advice is not so good, but that is how I'd done it.
Now, I'm kinda interested to know too if there is a less hacky methods to determine a volume of written data per device per day.

-- 
With kindest regards, Alexander.

⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀⠀⠀⠀