|
1 | 1 | # words_extractor |
2 | 2 |
|
3 | | -Example of text file parsing in Python, Golang, Elixir, Rust, Crystal and Julia |
4 | | - |
5 | | -Text source: 79.4MB in 30 files |
6 | | - |
7 | | -- Rust 1.51.0 (parallel) with sorting: 0.91s, without sorting: 0.49s |
8 | | -- Python 3.9.5 (parallel) with sorting: 3.42s, without sorting: 2.47 |
9 | | -- Crystal 1.0.0 (parallel) with sorting: 5.34s, withouts sorting: 2.48 |
10 | | -- Go 1.16.4 (parallel) with sorting: 6.41s, without sorting: 3.75s |
11 | | -- Rust 1.51.0 with sorting: 7s, without sorting: 5s (no parallelism) |
12 | | -- Julia 1.6.1 (8 threads) 8.7s, (1 thread) 9.7s without sorting |
13 | | -- Python 3.9.5 with sorting: 10s, without sorting 8.32s (no parallelism) |
14 | | -- Crystal 1.0.0 with sorting: 13s, without sorting: 7s (no parallelism) |
15 | | -- Go 1.16.4 with sorting: 21s, without sorting: 11s (no parallelism) |
16 | | -- Elixir 1.12 (parallel) with sorting: 33s (without release build) |
17 | | - |
18 | | -macOS 11.3.1, MacBook Pro (Retina, 15-inch, Late 2013) |
19 | | - |
20 | | -Python |
21 | | - |
22 | | -```bash |
23 | | -cd words_extractor_py |
24 | | -python words.py |
25 | | -``` |
26 | | - |
27 | | -Rust |
28 | | - |
29 | | -``` |
30 | | -cd words_extractor_rs |
31 | | -cargo build --release |
32 | | -target/release/words_extractor_rs |
33 | | -
|
34 | | -Golang |
35 | | -``` |
36 | | - |
37 | | -cd words_extractor_go |
38 | | -make build |
39 | | -GOGC=2000 ./main |
40 | | - |
41 | | -Crystal |
42 | | - |
43 | | -``` |
44 | | -cd words_extractor_cr |
45 | | -crystal build --release -Dpreview_mt src/fast_words_cr.cr -o main |
46 | | -CRYSTAL_WORKES=8 ./main |
47 | | -``` |
48 | | - |
49 | | -Julia |
50 | | - |
51 | | -``` |
52 | | -julia -t 8 src/words_extractor_jl.jl |
53 | | -``` |
54 | | - |
55 | | -Elixir |
56 | | - |
57 | | -``` |
58 | | -cd words_extractor_ex |
59 | | -mix run -e "WordsExtractor.run" |
60 | | -``` |
61 | | - |
62 | | -## Running Python |
63 | | - |
64 | | -1. Install the latest Python 3.9.5 |
65 | | -2. Create venv and dependencies |
66 | | - |
67 | | -```bash |
68 | | -cd words_extractor_py |
69 | | -python -m venv venv |
70 | | -source venv/bin/activate |
71 | | -pip install -r requirements.txt |
72 | | -``` |
73 | | - |
74 | | -3. Run the code |
75 | | - |
76 | | -```bash |
77 | | -python words_parallel.py |
78 | | -``` |
| 3 | +| | | | | | |
| 4 | +|--- |--- |--- |--- |--- | |
| 5 | +| | | | | | |
| 6 | +| | | | | | |
| 7 | +| | | | | | |
| 8 | + |
| 9 | +Example of a text file parsing in several programming languages |
| 10 | + |
| 11 | +MacOS 12.2 |
| 12 | +Rust 1.58.1 |
| 13 | +MBP 16" 64GB 2TB M1Max 10 cores |
| 14 | +Tested on 123 files (504MB) |
| 15 | + |
| 16 | +Results: |
| 17 | + |
| 18 | +1. Rust 1.58.1 -> 0.3521 s |
| 19 | +2. Ruby 3.1 with Parallel -> 2.0542 s |
| 20 | +3. Python 3.10.2 with multiprocessing -> 2.9403 s |
| 21 | +4. Crystal 1.3.2 with channels -> 6.0035 s |
| 22 | +5. Go 1.18beta1 with waitgroup -> 7.2166 s |
0 commit comments