- This topic has 18 replies, 6 voices, and was last updated Dec 4-8:19 am by ModdIt.
-
Posts
-
BobC
I found out I have another bad USB flashdrive. I used live-usb-maker to create it, and everything went ok, but it wouldn’t boot. I did it again 2 more times with similar results. I then used dd to write an iso to the flashdrive and then used a command I found to check to see if it was written correctly. As you can see, you need root authority to run it. The third time I had used a different flashdrive, and on that one the result was valid.
Sorry, not sure how to enter this code as the backticks are messing up the CODE tag and /CODE tag
$ cmp -n
stat -c '%s' endeavouros-2020.09.20-x86_64.isoendeavouros-2020.09.20-x86_64.iso /dev/sdd
cmp: /dev/sdd: Permission denied
bobc@xps15-7559:/media/bobc/BIGData/iso
$ sudo cmp -nstat -c '%s' endeavouros-2020.09.20-x86_64.isoendeavouros-2020.09.20-x86_64.iso /dev/sdd
endeavouros-2020.09.20-x86_64.iso /dev/sdd differ: char 17707023, line 67892
bobc@xps15-7559:/media/bobc/BIGData/iso
$ sudo cmp -nstat -c '%s' endeavouros-2020.09.20-x86_64.isoendeavouros-2020.09.20-x86_64.iso /dev/sdd
bobc@xps15-7559:/media/bobc/BIGData/isoI got the code from this thread: https://unix.stackexchange.com/questions/75483/how-to-check-if-the-iso-was-written-to-my-usb-stick-without-errors
My question is, I know how to check it if its written by dd, but how can I check it if its written by live-usb-maker in non-dd mode? And could these checks either be done optionally or by default automatically by the program at the end as part of the process instead of me trying to figure it out? I do know that antiX has a check function when booting its USB, but it would be much better to know when writing it that it was able to write, but that the flashdrive was not valid.
- This topic was modified 2 years, 5 months ago by BobC.
I used live-usb-maker to write antix full to the suspect USB, and it says it was successful, but when I boot it and take F4 and select checkmd5 it fails. I am running the checkfs now.
I’m just asking if live-usb-maker could test what it wrote to tell me it was wrong then, rather than find it out later? Most people won’t check it, and will run into problems, and think the problem was with antiX rather than the flashdrive.
ModdIt
Up to now I thought the writer would check the image written is same as source.
I asked about the format method because as I mentioned several drives showed same symptoms
but fixed by the reformatting. The drives I fixed all had a second partition
which was problematic for the install but no errors thrown.All this stuff helps me figure out what to do when things go wrong and the phone rings :-).
Brian Masinick
I usually clear a USB drive before writing new content unless I am using Ventoy to hold multiple images.
I have not encountered this problem fortunately but I will take note of it in case I see it in the future.
--
Brian MasinickAnonymous
As you can see, you need root authority to run it.
Use of the command doesn’t require elevated permissions.
Apparently the mount, or the file(s) slated for comparison, were readable only by root.how can I check it if its written by live-usb-maker in non-dd mode?
The live-usb-maker tool diligently calls “sync” after its write operations.
A quick check (search in page: sync) https://github.com/BitJam/live-usb-maker/blob/master/live-usb-maker finds sync is mentioned 16 times throughout the script.Also, @masinick “I usually clear a USB drive before writing new content“:
refer to the clear_partition() and clear_filesystem() functions within live-usb-maker ~~ it attends to this detail.
The in-app menu of live-usb-maker additionally provides on-demand access to the “clear_partition” operation.> how can I check
If you paste the following into your conkyrc…
${execi 5 iostat -dp | tail -n +3 | grep -v loop | awk '{ printf "%-7s %7s %8s\n", $1, $5, $6}'}…it will (per “5” in my example above) provide updated output every 5secs, ala
Device: kB_read kB_wrtn sda 4207 0 sda1 3635 0 sdb 1773573 119 sdb1 1773033 119 sdc 824743 4341 sdc1 824215 4341providing visual confirmation of kb_wrtn value changes as they occur
I know how to check it if its written by dd, but how can I check it if its written by live-usb-maker in non-dd mode?
Aw, i missed or misread the gist of your question ~~ in non-dd mode
but I’m not gonna erase the bit I’ve already typed here.What is “it”? I guess you mean iso file.
Skimming through the live-usb-maker code (itza bash script, plus external sourced scripts)
As far as I can tell, unless one has requested “clone” mode, an iso file will be handled the via copy_files_spec routine, @L2297, which invokes “cp” command (not “dd”)And could these checks either be done optionally or by default automatically by the program at the end as part of the process instead of me trying to figure it out?
AFAIK, the checks are an intrinsic part of live-usb-maker’s operation.
In fact, as I recall, someone had requested a way to bypass the checks, b/c checking huge files, on old/slow hardware was painfully slow… and BitJam responded by adding an option ––force=nomd5 to accommodate that request.No, the program did not check if it had written the usb correctly for an antix iso. It said it was successful, but when i shutdown and rebooted to it, the checkmd5 failed and when I tried again the checkfs crashed the computer.
If the program had an option to check, that would be great. I don’t have any idea how to check an antix iso manually, or I could have done that before shutting down.
^— too many words.
said differently:
If live-usb-maker checks to ensure source<->dest *.iso md5sums match (and I believe it does), then its job is done. It (we) can’t predict whether faulty flash media will result in future erratic md5sum results.very strange. My “too many words” post was posted after this long post and the up arrow was intended to point back to this message. At this point, although I am able to edit THIS post, I cannot edit/delete that other post.
No, the program did not check if it had written the usb correctly for an antix iso.
It said it was successful, but when i shutdown and rebooted to it,
the checkmd5 failed and when I tried again the checkfs crashed the computer.That sounds like it could be due to a missed call to sync (which would be a fixable bug)
leaving you with the incorrect impression that the copy operation had completed & safe to shutdown.If you post the exact commandline options you used and/or the menu selections you chose, we can trace the workflow, checking for a missed sync call (but I doubt that’s what is happening, b/c a sync op is called at the foot of the script’s main() routine).
Alternatively, for testing/demonstration, could create a wrapper fn instead of calling sync directly, pop a yad dialog to report “script called sync here, and the sync operation has completed”
If the program had an option to check, that would be great.
I don’t have any idea how to check an antix iso manually, or I could have done that before shutting down.md5sum /path/tofileA /path/to/fileB
whenever “have no idea” strikes, apropos is your everpresent friend.
apropos integrity
apropos verify
apropos sum
apropos md5
aha…
man md5sumWhether or not a clicky menu option, specific to (re)checking iso md5sum on-demand, will be added into the live-usb-maker script is not my decision. As with several of the antiX utilities, I ‘spect the current version of live-usb-maker is not posted to the gitlab project repo, so I can’t act on your suggestion by patching to add the menu entry and sending up a suggested merge request.
After trying a few times with live-usb-maker, I tried the same flashdrive with a program called Etcher with the non-antix iso and it reported that the USB was faulty. Obviously that program can’t create the USB for an antix iso, and I had to run it from an appimage, so I would much rather use live-usb-maker, but was just wondering if live-usb-maker could also be tweaked to optionally also notice that the USB was bad.
Thanks for deeper insights skidoo.
Any ideas welcome as to why live sticks failed to boot repeatedly using the system
tools but could be used after reformatting with. sudo mkfs.vfat /dev/sdXX.I am wondering if live usb maker does a quick format. The full format would
I assume cause the stick controller to mark any bad blocks, maybe quick
would not do that.The latest spate of problems was with 3 Intenso 32GB USB 3 sticks.
All had boot failure until fully formatted.
I had doublechecked the downloaded iso files before writing. No errors
from usb writing tool.From here what helped me, hopefully clearly understandable.
All sticks were checked with F3 Tool after a full format to vfat using quoted command..
Capacity and state tested OK.
An antiX ISO was written on each stick. Runit Full
other two normal full latest 64 bit. All checksums ok.
All drives are used for live sticks and now still working ok.
All have been remastered several times.@ BobC
A while back I posted on checking flash media with F3
rather than another big post just a pointer to the website.
The application is available in the repos.
Website
https://fight-flash-fraud.readthedocs.io/en/stable/Maybe you can gain some insights by using the tools on your stick.
olsztyn
said differently:
If live-usb-maker checks to ensure source<->dest *.iso md5sums match (and I believe it does), then its job is done. It (we) can’t predict whether faulty flash media will result in future erratic md5sum results.If I can enter MHO here:
– The Live USB Maker’s progress log includes at the end md5 integrity check of linuxfs and initrdr (skidoo – please correct me if I am misnterpreting).
– Performing the above it considers verification of the output done. In my understanding what BobC is suggesting is to verify reliability of USB flash before performing creation of Live USB, which it does not do, although some other programs might do (Perhaps Balena Etcher, but my understanding was Balena Etcher checks ISO, I did not know about checking reliability of USB flash.
– I created many Live-USB-Maker made copies and some resulted in failure reported by Live-USB-Maker at the end but that was not because flash was unreliable but rather that running Live-USB-Maker I had my laptop plugged into docking station. Took me some time to discover that if I run it in the same laptop but not plugged into docking station it runs fine on the same flash. So in my case it had nothing to do with flash reliability, although at first I was quick to fault Live-USB-Maker for this issue…
– I must have about 10 USB sticks with antiX on them and not one seems unreliable. I am using mostly Cruzer FIT, just a little plug USB2 most of them. They proved fast and reliable. On the opposite PNY 16G sticks proved extremely slow, so using just for antiX backup, but also no reliability issue that I have detected. This puzzles me that BobC has lots of flash reliability issues, could be some CN knock-offs as bad experience I had.
– To close my rambling on flash reliability issues BobC discovered, this should not happen often nowadays unless some eBay CN stuff marked 10x capacity they are able to hold. If you stick to typical brands, even from CN, then it should not happen and perhaps requiring Live-USB-Maker to verify reliability of flash could be an overkill and not assuring that such flash would not fail after test anyway…Just my two cents…
Live antiX Boot Options (Previously posted by Xecure):
https://antixlinuxfan.miraheze.org/wiki/Table_of_antiX_Boot_ParametersThe task of stress testing the media beforehand… seems, to me, beyond of scope of the live-usb-maker utility.
> at the end as part of the process instead of me trying to figure it out?
As mentioned, it already does so. For added peace of mind, I’ll suggest calling live-usb-maker from a wrapper script, running scripted command “md5sum /path/to/generated.iso” NN times as a followup so that you can verify a consistent readback result. Even so — as illustrated by olsztyn’s example (situationally undervolted USB port?) — we can’t predict what readback inconsitencies may arise during a later session.
Yes, a long time ago, on small USB’s once when I had problems with cheap ones being fakes, h2w on Win and then f3 on Linux was able to test them, but with these newer, bigger ones it was taking a day. It’s been running 2 hours now. I am hoping it will be done tomorrow sometime. That is to check 4 gb flashdrive. A 32 gb one would take about a week, but I gave up the last time I tried it. I found a script called log-f3wr and had to modify it to get it to work. It does an f3write, then f3read and logs the output to a file. This is my modified version.
#!/usr/bin/env bash LOG=$1 f3write "${@:2}" 2>&1 | tee -a "${LOG}" && \ echo -e "\n\n" >> "${LOG}" && \ f3read "${@:2}" 2>&1 | tee -a "${LOG}"I showed at the beginning what I found to check a flashdrive written with a “normal” iso that was written using dd, and it works. It takes a minute or two to run.
With the antiX iso it doesn’t normally use dd to write, so the test used to check dd files written to read it doesn’t work. I understand how to calculate an md5 or sha512 of a file. My problem is that I don’t know what to compare to what. For example antiX creates and writes 2 partitions, one with ANTIX-UEFI, and the other with antiX-Live-usb. There must be some way to find out if when I read them, the data is the same as what was written or not, and if not, then there are differences, and its “invalid” in my book. Maybe I just didn’t say that clearly.
I guess I will try again looking at the script to try to see what it wrote to where. Maybe that will provide a clue. I don’t have bad flashdrives often, but it bothers me that I spent 4 or 5 hours failinag at trying to boot it because I didn’t know it was bad, and had to resort to installing Etcher to figure it out (or I could have used Windows and maybe Rufus would have told me). It previously had held my antiX backup, so I’m glad I found out it was bad before I needed that.
I just noticed that the script does not “echo 1 > /proc/sys/vm/drop_caches”
after calling the sync command.
I’m wondering whether it would beneficial to do so, prior to checking md5sum for the copied file(s).> to see what it wrote to where
sudo apt install strace (I don’t recall whether it is preinstalled in the distro)
sudo strace -e trace=open,write -o /tmp/lum_strace_logfile.txt live-usb-maker
(sumpinlikethat. I didn’t test that exact commandstring)> problem is that I don’t know what to compare to what
If you post the strace log to pastebin (hmmm, may be too large), I’ll take a look.
The list of “source” files, I expect will vary depending on your selected live-usb-maker menu option(s). If cloning an existing liveUSB device, the copy operations would be pretty straightforward. If creating from a running system, the utility might be extracting some of the various “source” copies from within archivefiles at runtime.
anticapitalista
*testing*
Philosophers have interpreted the world in many ways; the point is to change it.
antiX with runit - leaner and meaner.
- You must be logged in to reply to this topic.
