Python Saving Output From A For Iteration And Subprocess For Checksum

The purpose of this script is to pull md5 checksum from each file of a directory as source and then (I'm working on that also) execute the script on the destination so validate it

Solution 1:

ah, someone asked for alternatives, there are of course :)

import logging
import hashlib
import os
outfile = "hash.log"
indir = "/Users/daniel/Sites/work"
logging.basicConfig(filename=outfile, filemode="w", format='%(message)s', level=logging.DEBUG)
for filename in (file for file inos.listdir(indir) ifos.path.isfile(file) andnot file.endswith("~")):
    with open(filename) as checkfile:
        logging.info(hashlib.md5(checkfile.read()).hexdigest())

i've been using something like this before.

what i like is using the logging module, because it makes things scalable, i don't have to keep a file open, or keep on opening it. the logger is highly configurable, but for just generating something like needed here, the simple setup is a one liner.

here i am not doing any console parsing, because i am using pythons hashlib to generate the file md5. now one could say, doing this could be slowing things down, but at least for the file sizes i usually encounter i had no problems so far.

would be interesting to test on larger files, otherwise the logging mechanism could also be used in your case. i only preferred hashlib back then, because i did not fancy parsing console output.

Solution 2:

You keep opening with w and overwriting , open with a to append.

The best way is to simply redirect stdout to a file object, something like:

defcheca_sum(x):
    withopen(archivo,'a') as outfile:
        check_call(["md5",x], stdout=outfile)

using check_call will raise a CalledProcessError for a non-zero exit status which you should handle accordingly.

To catch the exception:

try:
     check_call(["md5sum", x], stdout=outfile)
  except CalledProcessError as e:
     print("Exception for {}".format(e.cmd))

Use a generator expression to get the files and if you want to ignore copies use not f.endswith("~"):

files = (f for f inos.listdir("/home/padraic") ifos.path.isfile(f) andnot f.endswith("~"))
for i in files:
    checa_sum(i)