|
| 1 | +What is sc2reader? |
| 2 | +==================== |
| 3 | + |
| 4 | +sc2reader is a python library for extracting information from various different |
| 5 | +Starcraft II resources. These resources currently include Replays, Maps, and |
| 6 | +Game Summaries; we will eventually include BNet profiles and possibly adapters |
| 7 | +to the more entrenched SCII sites like sc2ranks. |
| 8 | + |
| 9 | +Our goal is to give anyone and everyone the power to construct their own |
| 10 | +tools, do their own analysis, and hack on their own Starcraft II projects |
| 11 | +under the open MIT license. Currently powering: |
| 12 | + |
| 13 | +* Websites: `ggtracker.com`_ |
| 14 | +* Research: `Build Order Classification` |
| 15 | + |
| 16 | +Our secondary goal is to become a reference implementation for people looking |
| 17 | +to implement parsers in other languages. There are currently implementations |
| 18 | +under development in: |
| 19 | + |
| 20 | +* C#: `sc2replay-csharp`_ (special thanks for v1.5 help) |
| 21 | +* C++: `sc2pp`_ |
| 22 | +* Javascript: `comsat`_ |
| 23 | +* PHP: `phpsc2replay`_ (the original open implementation!) |
| 24 | + |
| 25 | +If you'd like your tool, site, project, or implementation listed above, drop |
| 26 | +us a line on our mailing list or stop by our #sc2reader IRC channel and say hi! |
| 27 | + |
| 28 | + |
| 29 | +Current Status |
| 30 | +================= |
| 31 | + |
| 32 | +sc2reader is currently capable of parsing varying levels of information out of |
| 33 | +the three primary resource types listed below. For a more detailed and exact |
| 34 | +description of the information that can be extracted please consult the |
| 35 | +`documentation`_ hosted on Read the Docs and packaged with the source. |
| 36 | + |
| 37 | + |
| 38 | +Replays |
| 39 | +------------- |
| 40 | + |
| 41 | +Almost all the basic contextual information can be extracted from any post-beta |
| 42 | +replays. This information includes: |
| 43 | + |
| 44 | +- Replay details (map, length, version, datetime, game type, game speed, ...) |
| 45 | +- Player details (name, race, team, color, bnet url, win/loss, ...) |
| 46 | +- Message details (text, time, player, target, pings, ...) |
| 47 | + |
| 48 | +Additional information can be parsed from ladder replays and replays from basic |
| 49 | +unmodded private "custom" games. This information includes: |
| 50 | + |
| 51 | +- Unit Selection and Hotkey events. |
| 52 | +- Resource Transfers and Requests (but not collection or unspent values!) |
| 53 | +- Unfiltered Unit commands (attack, move, train, build, psi storm, etc) |
| 54 | +- Camera Movements for all players and observers. |
| 55 | + |
| 56 | +In some cases, further information can be extracted from this raw information: |
| 57 | + |
| 58 | +- All unit selections and hotkey values for every frame of the game. |
| 59 | +- APM/EPM and its untold variations. |
| 60 | + |
| 61 | +We are in the process of building data dictionaries for all the SC2 units and |
| 62 | +abilities which should enable much more creative and robust analysis of the |
| 63 | +raw event stream found in replay files. |
| 64 | + |
| 65 | + |
| 66 | +Maps |
| 67 | +------- |
| 68 | + |
| 69 | +Maps are currently parsed in the most minimal way possible and are an area for |
| 70 | +big improvement in the future. Currently the Minimap.tga packaged with the map |
| 71 | +is made available along with Name, Author, Description for ONLY enUS localized |
| 72 | +SCII map files. More robust Map meta data extraction will likely be added in |
| 73 | +future releases. |
| 74 | + |
| 75 | +There's a lot more in here to be had for the adventurous. |
| 76 | + |
| 77 | + |
| 78 | +Game Summaries |
| 79 | +----------------- |
| 80 | + |
| 81 | +Tons of data parsed. Thank you Prillan and others from `Team Liquid`_. |
| 82 | + |
| 83 | +* Lobby Properties (game speed, game type, ...) |
| 84 | +* Player Information (Race, Team, Result, bnet info, ...) |
| 85 | +* Player Graphs & Stats (Army Graph, Income Graph, Avg Unspent Resources, ...) |
| 86 | +* URLs to map localization files and images |
| 87 | +* Player build orders up to 64 (real) actions |
| 88 | + |
| 89 | +This isn't super reliable yet and s2gs files may fail during processing. We've |
| 90 | +figured out the basic common structure and where the information is stored but |
| 91 | +the data structure sometimes can't be processed with current techniques and it |
| 92 | +seems as though different s2gs files can contain radically different amounts |
| 93 | +of information based on some unknown factors. |
| 94 | + |
| 95 | +It is likely that s2gs file support will be improved in future releases. |
| 96 | + |
| 97 | + |
| 98 | +Example Usage |
| 99 | +===================== |
| 100 | + |
| 101 | +To demonstrate how you might use sc2reader in practice I've included some short |
| 102 | +contrived scripts below. For more elaborate examples, checkout the docs and the |
| 103 | +sc2reader.scripts package on Github. |
| 104 | + |
| 105 | +Downloading Maps |
| 106 | +-------------------- |
| 107 | + |
| 108 | +Save all the minimaps for all the games you've ever played:: |
| 109 | + |
| 110 | + import sc2reader, os, sys |
| 111 | + |
| 112 | + if not os.path.exists('minimaps'): |
| 113 | + os.makedirs('minimaps') |
| 114 | + |
| 115 | + # Only load details file (level 1) and fetch the map file from bnet |
| 116 | + for replay in sc2reader.load_replays(sys.argv[1:], load_map=True, load_level=1): |
| 117 | + minimap_path = os.path.join('minimaps', replay.map_name.replace(' ','_')+'.tga') |
| 118 | + if not os.path.exists(minimap_path): |
| 119 | + with open(minimap_path, 'w') as file_out: |
| 120 | + file_out.write(replay.map.minimap) |
| 121 | + print "Saved Map: {0} [{1}]".format(replay.map_name, replay.map_hash) |
| 122 | + |
| 123 | + |
| 124 | +Organizing Replays |
| 125 | +---------------------- |
| 126 | + |
| 127 | +Organizing your 1v1 replays by race played and matchup and sortable by length:: |
| 128 | + |
| 129 | + import sc2reader, os, shutil, sys |
| 130 | + |
| 131 | + sorted_base = 'sorted' |
| 132 | + path_to_replays = 'my/replays' |
| 133 | + |
| 134 | + for replay in sc2reader.load_replays(sys.argv[1], load_level=1): |
| 135 | + if replay.real_type != '1v1': |
| 136 | + continue |
| 137 | + |
| 138 | + try: |
| 139 | + me = replay.player.name('ShadesofGray') |
| 140 | + you = team[(me.team.number+1)%2].players[0] |
| 141 | + |
| 142 | + matchup = "{}v{}".format(me.play_race[0], you.play_race[1]) |
| 143 | + |
| 144 | + sorted_path = os.path.join(sorted_base,me.play_race[0],matchup) |
| 145 | + if not os.path.exists(sorted_path): |
| 146 | + os.makedirs(sorted_path) |
| 147 | + |
| 148 | + filename = "{0} - {1}".format(replay.game_length, replay.filename) |
| 149 | + src = os.join(path_to_replays,replay.filename) |
| 150 | + dst = os.join(sorted_path, filename) |
| 151 | + shutil.copyfile(src, dst) |
| 152 | + |
| 153 | + except KeyError as e: |
| 154 | + continue # A game I didn't play in! |
| 155 | + |
| 156 | + |
| 157 | +Installation |
| 158 | +================ |
| 159 | + |
| 160 | +From PyPI (stable) |
| 161 | +--------------------- |
| 162 | + |
| 163 | +Install from the latest release on PyPI with pip:: |
| 164 | + |
| 165 | + pip install sc2reader |
| 166 | + |
| 167 | +or easy_install:: |
| 168 | + |
| 169 | + easy_install sc2reader |
| 170 | + |
| 171 | +or with setuptools (specify a valid x.x.x):: |
| 172 | + |
| 173 | + wget http://pypi.python.org/packages/source/s/sc2reader/sc2reader-x.x.x.tar.gz |
| 174 | + tar -xzf sc2reader-x.x.x.tar.gz |
| 175 | + cd sc2reader-x.x.x |
| 176 | + python setup.py install |
| 177 | + |
| 178 | + |
| 179 | +From Github |
| 180 | +-------------------------- |
| 181 | + |
| 182 | +Github master is generally stable with development branches more unstable. |
| 183 | + |
| 184 | +Install from the latest source on github with pip:: |
| 185 | + |
| 186 | + pip install -e git+git://github.com/GraylinKim/sc2reader#egg=sc2reader |
| 187 | + |
| 188 | +or with setuptools:: |
| 189 | + |
| 190 | + wget -O sc2reader-master.tar.gz https://github.com/GraylinKim/sc2reader/tarball/master |
| 191 | + tar -xzf sc2reader-master.tar.gz |
| 192 | + cd sc2reader-master |
| 193 | + python setup.py install |
| 194 | + |
| 195 | + |
| 196 | +For Contributors |
| 197 | +------------------- |
| 198 | + |
| 199 | +Contributors should install from an active git repository using setuptools in |
| 200 | +`develop`_ mode. This will install links to the live code so that local edits |
| 201 | +are available to external modules automatically:: |
| 202 | + |
| 203 | + git clone https://github.com/GraylinKim/sc2reader.git |
| 204 | + cd sc2reader |
| 205 | + python setup.py develop |
| 206 | + |
| 207 | +Contributions can be sent via pull request or by `mailing list`_ with attached |
| 208 | +patch files. It is highly recommended you get in touch with us before working |
| 209 | +on patches. |
| 210 | + |
| 211 | + |
| 212 | +Testing |
| 213 | +------------------- |
| 214 | + |
| 215 | + mkdir testcache |
| 216 | + GGFACTORY_CACHE_DIR=testcache py.test |
| 217 | + |
| 218 | + |
| 219 | +Community |
| 220 | +============== |
| 221 | + |
| 222 | +sc2reader has a small but growing community of people looking to make tools and |
| 223 | +websites with Starcraft II data. If that sounds like something you'd like to be |
| 224 | +a part of please join our underused `mailing list`_ and start a conversation |
| 225 | +or stop by #sc2reader on FreeNode and say 'Hi'. We have members from all over |
| 226 | +Europe, Australia, and the United States currently, so regardless of the time, |
| 227 | +you can probably find someone to talk to. |
| 228 | + |
| 229 | + |
| 230 | +Issues and Support |
| 231 | +===================== |
| 232 | + |
| 233 | +We have an `issue tracker`_ on Github that all bug reports and feature requests |
| 234 | +should be directed to. We have a `mailing list`_ with Google Groups that you can |
| 235 | +use to reach out for support. We are generally on FreeNode in the #sc2reader |
| 236 | +and can generally provide live support and address issues there as well. |
| 237 | + |
| 238 | + |
| 239 | +Acknowledgements |
| 240 | +======================= |
| 241 | + |
| 242 | +Thanks to all the awesome developers in the SC2 community that helped out |
| 243 | +and kept this project going. Special thanks to the people of the awesome |
| 244 | +`phpsc2replay`_ project whose public documentation and source code made |
| 245 | +starting this library possible and to sc2replay-csharp for setting us |
| 246 | +straight on game events parsing and assisting with our v1.5 upgrade. |
| 247 | + |
| 248 | +.. _Build Order Classification: https://github.com/grahamjenson/sc2reader |
| 249 | +.. _ggtracker.com: http://ggtracker.com |
| 250 | +.. _PyPy: http://pypy.org/ |
| 251 | +.. _sc2pp: https://github.com/zsol/sc2pp |
| 252 | +.. _sc2replay-csharp: https://github.com/ascendedguard/sc2replay-csharp |
| 253 | +.. _comsat: https://github.com/tec27/comsat |
| 254 | +.. _phpsc2replay: http://code.google.com/p/phpsc2replay/ |
| 255 | +.. _Team Liquid: http://www.teamliquid.net/forum/viewmessage.php?topic_id=330926 |
| 256 | +.. _develop: http://peak.telecommunity.com/DevCenter/setuptools#development-mode |
| 257 | +.. _documentation: http://sc2reader.rtfd.org/ |
| 258 | +.. _mailing list: http://groups.google.com/group/sc2reader |
| 259 | +.. _developers mailing list: http://groups.google.com/group/sc2reader-dev |
| 260 | +.. _phpsc2replay: http://code.google.com/p/phpsc2replay/ |
| 261 | +.. _issue tracker: https://github.com/GraylinKim/sc2reader/issues |
0 commit comments