File tree Expand file tree Collapse file tree 4 files changed +103
-0
lines changed
Expand file tree Collapse file tree 4 files changed +103
-0
lines changed Original file line number Diff line number Diff line change 1+
2+ source 'https://rubygems.org'
3+ git_source ( :github ) { |repo | "https://github.com/#{ repo } .git" }
4+
5+ ruby '3.1.0'
6+
7+ gem 'rubocop' , require : false
8+ gem 'parallel'
9+ # gem 'irbtools', require: 'irbtools/binding'
10+ gem 'pry'
11+ gem 'pry-doc'
12+ gem 'pry-gem' , '~> 1.0'
Original file line number Diff line number Diff line change 1+ GEM
2+ remote: https://rubygems.org/
3+ specs:
4+ ast (2.4.2 )
5+ coderay (1.1.3 )
6+ method_source (1.0.0 )
7+ parallel (1.21.0 )
8+ parser (3.1.0.0 )
9+ ast (~> 2.4.1 )
10+ pry (0.14.1 )
11+ coderay (~> 1.1 )
12+ method_source (~> 1.0 )
13+ pry-doc (1.3.0 )
14+ pry (~> 0.11 )
15+ yard (~> 0.9.11 )
16+ pry-gem (1.0.0 )
17+ pry (~> 0.12 )
18+ rainbow (3.1.1 )
19+ regexp_parser (2.2.0 )
20+ rexml (3.2.5 )
21+ rubocop (1.25.1 )
22+ parallel (~> 1.10 )
23+ parser (>= 3.1.0.0 )
24+ rainbow (>= 2.2.2 , < 4.0 )
25+ regexp_parser (>= 1.8 , < 3.0 )
26+ rexml
27+ rubocop-ast (>= 1.15.1 , < 2.0 )
28+ ruby-progressbar (~> 1.7 )
29+ unicode-display_width (>= 1.4.0 , < 3.0 )
30+ rubocop-ast (1.15.1 )
31+ parser (>= 3.0.1.1 )
32+ ruby-progressbar (1.11.0 )
33+ unicode-display_width (2.1.0 )
34+ webrick (1.7.0 )
35+ yard (0.9.27 )
36+ webrick (~> 1.7.0 )
37+
38+ PLATFORMS
39+ ruby
40+
41+ DEPENDENCIES
42+ parallel
43+ pry
44+ pry-doc
45+ pry-gem (~> 1.0 )
46+ rubocop
47+
48+ RUBY VERSION
49+ ruby 3.1.0p0
50+
51+ BUNDLED WITH
52+ 2.3.5
Original file line number Diff line number Diff line change 1+ ```
2+ ruby words.rb
3+ ```
4+
5+ MacOS 12.2
6+ Ruby 3.1
7+ MBP 16" M1Max 10 cores
8+ Total files: 123
9+ Total size: 504 MB
10+ Total time: 2.0542 s
Original file line number Diff line number Diff line change 1+ require 'yaml'
2+ require 'parallel'
3+ require 'etc'
4+ require 'fileutils'
5+
6+ outdir = 'words'
7+ start = Time . now
8+
9+ FileUtils . rm_rf ( outdir )
10+ Dir . mkdir ( outdir )
11+
12+ sorted = false
13+
14+ paths = Dir [ '../data/pl/**/*.yml' ]
15+
16+ Parallel . each ( paths , in_processes : Etc . nprocessors ) do |yaml_path |
17+ meta = YAML . load_file ( yaml_path )
18+ words = IO . read ( yaml_path . gsub ( '.yml' , '.txt' ) ) . downcase . strip . split ( /[^\p {word}]+/ ) . uniq
19+ if sorted
20+ words = words . sort
21+ end
22+ outpath = "#{ outdir } /#{ meta [ 'lang' ] } -#{ meta [ 'code' ] } .txt"
23+ puts outpath
24+ File . write ( outpath , words . join ( "\n " ) )
25+ end
26+
27+ secs = Time . now - start
28+ puts "Total time: #{ secs } s"
29+
You can’t perform that action at this time.
0 commit comments