OLD | NEW |
1 Here's a rough walkthrough of how this works. The ultimate output file is | 1 Here's a rough walkthrough of how this works. The ultimate output file is |
2 database.filtered.json. | 2 database.filtered.json. |
3 | 3 |
| 4 full_run.sh executes all of the scripts in the correct order. |
| 5 |
4 search.js | 6 search.js |
5 - read data/domTypes.json | 7 - read data/domTypes.json |
6 - for each dom type: | 8 - for each dom type: |
7 - search for page on www.googleapis.com | 9 - search for page on www.googleapis.com |
8 - write search results to output/search/<type>.json | 10 - write search results to output/search/<type>.json |
9 . this is a list of search results and urls to pages | 11 . this is a list of search results and urls to pages |
10 | 12 |
11 crawl.js | 13 crawl.js |
12 - read data/domTypes.json | 14 - read data/domTypes.json |
13 - for each dom type: | 15 - for each dom type: |
(...skipping 25 matching lines...) Expand all Loading... |
39 - all sorts of shenanigans to actually pull the content out of the html | 41 - all sorts of shenanigans to actually pull the content out of the html |
40 - build a JSON object with the results | 42 - build a JSON object with the results |
41 - do a postmessage with that object so extractRunner.js can pull it out | 43 - do a postmessage with that object so extractRunner.js can pull it out |
42 | 44 |
43 - run postProcess.dart | 45 - run postProcess.dart |
44 - go through the results for each type looking for the best match | 46 - go through the results for each type looking for the best match |
45 - write output/database.html | 47 - write output/database.html |
46 - write output/examples.html | 48 - write output/examples.html |
47 - write output/obsolete.html | 49 - write output/obsolete.html |
48 - write output/database.filtered.json which is the best matches | 50 - write output/database.filtered.json which is the best matches |
OLD | NEW |