Skip to content

Commit dba6624

Browse files
author
Jaroslaw Zabiello
committed
update main README
1 parent 1947871 commit dba6624

File tree

1 file changed

+20
-76
lines changed

1 file changed

+20
-76
lines changed

README.md

Lines changed: 20 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -1,78 +1,22 @@
11
# words_extractor
22

3-
Example of text file parsing in Python, Golang, Elixir, Rust, Crystal and Julia
4-
5-
Text source: 79.4MB in 30 files
6-
7-
- Rust 1.51.0 (parallel) with sorting: 0.91s, without sorting: 0.49s
8-
- Python 3.9.5 (parallel) with sorting: 3.42s, without sorting: 2.47
9-
- Crystal 1.0.0 (parallel) with sorting: 5.34s, withouts sorting: 2.48
10-
- Go 1.16.4 (parallel) with sorting: 6.41s, without sorting: 3.75s
11-
- Rust 1.51.0 with sorting: 7s, without sorting: 5s (no parallelism)
12-
- Julia 1.6.1 (8 threads) 8.7s, (1 thread) 9.7s without sorting
13-
- Python 3.9.5 with sorting: 10s, without sorting 8.32s (no parallelism)
14-
- Crystal 1.0.0 with sorting: 13s, without sorting: 7s (no parallelism)
15-
- Go 1.16.4 with sorting: 21s, without sorting: 11s (no parallelism)
16-
- Elixir 1.12 (parallel) with sorting: 33s (without release build)
17-
18-
macOS 11.3.1, MacBook Pro (Retina, 15-inch, Late 2013)
19-
20-
Python
21-
22-
```bash
23-
cd words_extractor_py
24-
python words.py
25-
```
26-
27-
Rust
28-
29-
```
30-
cd words_extractor_rs
31-
cargo build --release
32-
target/release/words_extractor_rs
33-
34-
Golang
35-
```
36-
37-
cd words_extractor_go
38-
make build
39-
GOGC=2000 ./main
40-
41-
Crystal
42-
43-
```
44-
cd words_extractor_cr
45-
crystal build --release -Dpreview_mt src/fast_words_cr.cr -o main
46-
CRYSTAL_WORKES=8 ./main
47-
```
48-
49-
Julia
50-
51-
```
52-
julia -t 8 src/words_extractor_jl.jl
53-
```
54-
55-
Elixir
56-
57-
```
58-
cd words_extractor_ex
59-
mix run -e "WordsExtractor.run"
60-
```
61-
62-
## Running Python
63-
64-
1. Install the latest Python 3.9.5
65-
2. Create venv and dependencies
66-
67-
```bash
68-
cd words_extractor_py
69-
python -m venv venv
70-
source venv/bin/activate
71-
pip install -r requirements.txt
72-
```
73-
74-
3. Run the code
75-
76-
```bash
77-
python words_parallel.py
78-
```
3+
| | | | | |
4+
|--- |--- |--- |--- |--- |
5+
| | | | | |
6+
| | | | | |
7+
| | | | | |
8+
9+
Example of a text file parsing in several programming languages
10+
11+
MacOS 12.2
12+
Rust 1.58.1
13+
MBP 16" 64GB 2TB M1Max 10 cores
14+
Tested on 123 files (504MB)
15+
16+
Results:
17+
18+
1. Rust 1.58.1 -> 0.3521 s
19+
2. Ruby 3.1 with Parallel -> 2.0542 s
20+
3. Python 3.10.2 with multiprocessing -> 2.9403 s
21+
4. Crystal 1.3.2 with channels -> 6.0035 s
22+
5. Go 1.18beta1 with waitgroup -> 7.2166 s

0 commit comments

Comments
 (0)