#4080 new enhancement

Increase chunk sizes to improve performance and parallelism

Reported by: itamarst Owned by:
Priority: normal Milestone: Performance and Benchmarking
Component: unknown Version: n/a
Keywords: Cc:
Launchpad Bug:

Description

  1. Processing data in larger chunk sizes can reduce overhead of all the little Python function calls for each chunk.
  2. Larger chunk sizes for functions that release GIL add additional opportunities for utilizing parallelism by shifting to a thread pool. Larger chunks of work are more efficient for this scheme.

Change History (0)

Note: See TracTickets for help on using tickets.