18

(For context, I'm basically referring to Python 3.12 "multiprocessing.Pool Vs. concurrent.futures.ThreadPoolExecutor"...)

Today I read that multiple cores (parallelism) help in CPU bound operations. Meanwhile, multiple threads (concurrency) is due when the tasks are I/O bound.

Is this correct? Anyone cares to elaborate for me?

At least from a theorethical standpoint. Of course, many real work has a mix of both, and I'd better start with profiling where the bottlenecks really are.

If serves of anything having a concrete "algorithm". Let's say, I have a function that applies a map-reduce strategy reading data chunks from a file on disk, and I'm computing some averages from these data, and saving to a new file.

you are viewing a single comment's thread
view the rest of the comments
[-] dwt@feddit.org 1 points 2 months ago

On python, because of the Gil, multi processing should always be preferred if possible.

[-] logging_strict@programming.dev 1 points 1 month ago

Also logging is not isolated. Bleeds all over the place. Which is a deal breaker

Not worth the endless time doing forensics

Agree! Lets stick with multiprocessing

one thread sounds nice. Lets do much more of that

this post was submitted on 14 Oct 2024
18 points (100.0% liked)

Python

6478 readers
1 users here now

Welcome to the Python community on the programming.dev Lemmy instance!

📅 Events

PastNovember 2023

October 2023

July 2023

August 2023

September 2023

🐍 Python project:
💓 Python Community:
✨ Python Ecosystem:
🌌 Fediverse
Communities
Projects
Feeds

founded 2 years ago
MODERATORS