This is the useful links I used to solve the issue I got when using multiprocssing in python.
-
python Multithreading does not work well on windows. multiprocess sometimes have issues too. One simple example is about importing the self-defined module with multiprocessing in another code in windows NT OS. It is necessary to distinguish between the parent and child process with
__main__
. Refer to This -
what can be pickled in python? Functions are only picklable if they are defined at the top level of the module. Refer to this
-
When define the multiprocessing funtion inside the
class
, I got the error like Can't picklewhen using multiprocessing Pool.map() . I try to solve it withcopy_reg
as mentioned in 2. Another way to solve it is to usepathos.multiprocessing
. But finally I find this is the easiest way to do it: define the__call__
method in the same class to call the function. This is the easiest way for me to solve it.