jrmiller82
/
pathfinder-2-sqlite-MIRROR
mirror of https://gitlab.com/jrmiller82/pathfinder-2-sqlite.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Redo file structure

merge-requests/21/head
Brian Haley 2019-08-08 22:23:38 -04:00 committed by Brian
parent 67b483c44b
commit 87cce020ba
7 changed files with 239 additions and 482 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
scrape/README.md
View File

@ -1,31 +0,0 @@
# Directory for scraping [aon2e](https://2e.aonprd.com/Sources.aspx?ID=1)
## Current languages
* Python
### Python requirements
1. Python 3.6.8
2. pip
3. pandas 0.24.2
4. requests 2.21.0
5. beautifulsoup4==4.8.0
6. time
7. re
#### Spells
| Column | Data type | Comments |
| ------ | ------ | ------ |
| name | string | |
| level | integer | 1 to 9 |
| traits | list of strings | None may be possible |
| source | string | Follows format 'Core Rulebook pg. ?' |
| traditions | list of strings | should not contain any empty lists |
| actions | list of integers or string | contains either a list of integers representing the number of actions or a string representing the time to cast error handling isn't complete |
| components | list of strings | currently only contains material, somatic or verbal |
| spell_range | string | |
| target | string | |
| description | string | TODO separators for blank lines and |
| save | string | |
| duration | string | |
| success | ? | list of dictionaries maybe? |
| heighten | ? | list of dictionaries maybe? |

3
scrape/requirements.txt
View File

@ -1,3 +0,0 @@
pandas==0.24.2
requests==2.21.0
beautifulsoup4==4.8.0

448
scrape/scrape.ipynb
View File

@ -1,448 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Scrape data from aon2e and generate csvs to import in to sqlite"
]
},
{
"cell_type": "code",
"execution_count": 173,
"metadata": {},
"outputs": [],
"source": [
"# Dependencies\n",
"import pandas as pd\n",
"from bs4 import BeautifulSoup as bs\n",
"import requests\n",
"import time\n",
"import re\n",
"\n",
"# Pandas config\n",
"pd.set_option('display.max_columns', None)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Ancestries TODO"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# url that contains all the links\n",
"url_ancestry = 'https://2e.aonprd.com/Ancestries.aspx?ID='\n",
"\n",
"# Empty list to store the ancestry data\n",
"ancestry = []\n",
"\n",
"# Make the request to the aon2e\n",
"response_ancestry = requests.get(f'{url_ancestry}1')\n",
"\n",
"# Use BS4 html parser to generate soup\n",
"soup_ancestry = bs(response_ancestry.text, 'html.parser')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Store the data needed from the soup\n",
"name = soup_ancestry.find(id='ctl00_MainContent_DetailedOutput').a.text\n",
"traits = [trait.a.text for trait in soup_ancestry.find_all(class_='trait')]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Raw description\n",
"description = soup_ancestry.find(id='ctl00_MainContent_DetailedOutput').text"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Animal Companions TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Animals (Rentals/Sales) TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Arcane Schools TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Arcane Thesis TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Archetypes TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Armor TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Backgrounds TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Bloodlines TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Champion Causes TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Champion Tenets TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Classes TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Class Kits TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Class Sample Builds TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Conditions TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deities TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Doctrines TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Domains TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Druidic Orders TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Equipment TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Familiar Abilities TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Feats TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Hazards TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Hunter's Edges TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Instincts TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Languages TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Muses TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Rackets TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Research Fields TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Rituals TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Rules TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Shields TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Skills TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Skills (General) TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Spells TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Traits TODO"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Weapons TODO"
]
},
{
"cell_type": "code",
"execution_count": 306,
"metadata": {},
"outputs": [],
"source": [
"# url that contains all the links\n",
"url_spells = 'https://2e.aonprd.com/Spells.aspx?ID='\n",
"\n",
"# Number of spells taken from https://2e.aonprd.com/Sources.aspx?ID=1\n",
"spell_number = 343\n",
"\n",
"# Make the request to the aon2e\n",
"response_spells = requests.get(f'{url_spells}{spell_number}')\n",
"\n",
"# Use BS4 html parser to generate soup\n",
"soup_spells = bs(response_spells.text, 'html.parser')\n",
"\n",
"# Select only the content\n",
"content = soup_spells.find(id='ctl00_MainContent_DetailedOutput')"
]
},
{
"cell_type": "code",
"execution_count": 323,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['1 minute']\n"
]
}
],
"source": [
"name, level = content.h1.text.replace(' ', '').split('Spell')\n",
"traits = [trait.a.text.strip() for trait in content.find_all(class_='trait')]\n",
"source = content.find(class_='external-link').text.strip()\n",
"traditions = [tradition.text.strip() for tradition\n",
" in content.find_all('a', href=re.compile(\"Tradition\"))]\n",
"\n",
"## Actions sections\n",
"actions = []\n",
"# Start at cast and then iterate over the next elements on the line\n",
"for e in content.find('b', text='Cast').next_siblings:\n",
" if e.name == 'br':\n",
" # If the end of the line is reached break the loop\n",
" break\n",
" elif e.name == 'hr':\n",
" # If a horizontal line is reached\n",
" break\n",
" try:\n",
" if e['alt']=='Single Action' and 'actiondark' in e['class']:\n",
" # If it's the single action icon\n",
" actions.append(1)\n",
" elif e['alt']=='Three Actions' and 'actiondark' in e['class']:\n",
" # If it is the three action icon\n",
" actions.append(3)\n",
" except TypeError:\n",
" # If there is no icon handle it as a string\n",
" if 'to' in e:\n",
" # If to exists it must be 1 to 3 actions\n",
" actions.append(2)\n",
" else:\n",
" actions.append(e.split(' (')[0].strip())\n",
" continue\n",
" except KeyError:\n",
" continue\n",
" \n",
"components = content.find('b', text='Cast').next.next \\\n",
" .replace('(', '').replace(')', ''). replace(',', '').split(' ')\n",
"components = [x for x in components if x\n",
" in ['material', 'somatic', 'verbal']]\n",
"\n",
"try:\n",
" # If the spell has a range\n",
" spell_range = content.find('b', text='Range').next.next.replace(';', '')\n",
"except AttributeError:\n",
" # If the spell doesn't have a range\n",
" spell_range = None\n",
"\n",
"target = content.find('hr').previous.strip()\n",
"\n",
"# save TODO\n",
"\n",
"# duration TODO\n",
"\n",
"## Description section\n",
"description = content.find('hr').next\n",
"print(actions)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

BIN
src/resources/chromedriver.exe 100644
View File

Binary file not shown.

67
src/weapons/melee.csv 100644
View File

@ -0,0 +1,67 @@
Name,Category,Price,Damage,Bulk,Hands,Group,Weapon Traits
Fist,Unarmed,,1d4 B,,1,Brawling,"Agile, Finesse, Nonlethal, Unarmed"
Clan Dagger,Simple,2 gp,1d4 P,L,1,Knife,"Agile, Dwarf, Parry, Uncommon, Versatile B"
Club,Simple,0,1d6 B,1,1,Club,Thrown 10 ft.
Dagger,Simple,2 sp,1d4 P,L,1,Knife,"Agile, Finesse, Thrown 10 ft., Versatile S"
Gauntlet,Simple,2 sp,1d4 B,L,1,Brawling,"Agile, Free-Hand"
Katar,Simple,3 sp,1d4 P,L,1,Knife,"Agile, Deadly d6, Monk, Uncommon"
Light Mace,Simple,4 sp,1d4 B,L,1,Club,"Agile, Finesse, Shove"
Longspear,Simple,5 sp,1d8 P,2,2,Spear,Reach
Mace,Simple,1 gp,1d6 B,1,1,Club,Shove
Morningstar,Simple,1 gp,1d6 B,1,1,Club,Versatile P
Sickle,Simple,2 sp,1d4 S,L,1,Knife,"Agile, Finesse, Trip"
Spear,Simple,1 sp,1d6 P,1,1,Spear,Thrown 20 ft.
Spiked Gauntlet,Simple,3 sp,1d4 P,L,1,Brawling,"Agile, Free-Hand"
Staff,Simple,0,1d4 B,1,1,Club,Two-Hand d8
Bastard Sword,Martial,4 gp,1d8 S,1,1,Sword,Two-Hand d12
Battle Axe,Martial,1 sp,1d8 S,1,1,Axe,Sweep
Bo Staff,Martial,2 sp,1d8 B,2,2,Club,"Monk, Parry, Reach, Trip"
Dogslicer,Martial,1 sp,1d6 S,L,1,Sword,"Agile, Backstabber, Finesse, Goblin, Uncommon"
Elven Curve Blade,Martial,4 gp,1d8 S,2,2,Sword,"Elf, Finesse, Forceful"
Falchion,Martial,3 gp,1d10 S,2,2,Sword,"Forceful, Sweep"
Filcher's Fork,Martial,1 gp,1d4 P,L,1,Spear,"Agile, Backstabber, Deadly d6, Finesse, Halfling, Thrown 20 ft., Uncommon"
Flail,Martial,1 gp,1d6 B,1,1,Flail,"Disarm, Sweep, Trip"
Glaive,Martial,2 gp,1d8 S,2,2,Polearm,"Deadly d8, Forceful, Reach"
Gnome Hooked Hammer,Martial,2 gp,1d6 B,1,1,Hammer,"Gnome, Trip, Two-Hand d10, Uncommon, Versatile P"
Greataxe,Martial,1 gp,1d12 S,2,2,Axe,Sweep
Greatclub,Martial,1 gp,1d10 B,2,2,Club,"Backswing, Shove"
Greatpick,Martial,2 gp,1d10 P,2,2,Pick,Fatal d12
Greatsword,Martial,2 gp,1d12 S,2,2,Sword,Versatile P
Guisarme,Martial,2 gp,1d10 S,2,2,Polearm,"Reach, Trip"
Halberd,Martial,2 gp,1d10 P,2,2,Polearm,"Reach, Versatile S"
Hatchet,Martial,4 sp,1d6 S,L,1,Axe,"Agile, Sweep, Thrown 10 ft."
Horsechopper,Martial,9 sp,1d8 S,2,2,Polearm,"Goblin, Reach, Trip, Uncommon, Versatile P"
Kama,Martial,1 gp,1d6 S,L,1,Knife,"Agile, Monk, Trip, Uncommon"
Katana,Martial,2 gp,1d6 S,1,1,Sword,"Deadly d8, Two-Hand d10, Uncommon, Versatile P"
Kukri,Martial,6 sp,1d6 S,L,1,Knife,"Agile, Finesse, Trip, Uncommon"
Lance,Martial,1 gp,1d8 P,2,2,Spear,"Deadly d8, Jousting d6, Reach"
Light Hammer,Martial,3 sp,1d6 B,L,1,Hammer,"Agile, Thrown 20 ft."
Light Pick,Martial,4 sp,1d4 P,L,1,Pick,"Agile, Fatal d8"
Longsword,Martial,1 gp,1d8 S,1,1,Sword,Versatile P
Main-gauche,Martial,5 sp,1d4 P,L,1,Knife,"Agile, Disarm, Finesse, Parry, Versatile S"
Maul,Martial,3 gp,1d12 B,2,2,Hammer,Shove
Nunchaku,Martial,2 sp,1d6 B,L,1,Club,"Backswing, Disarm, Finesse, Monk, Uncommon"
Orc Knuckle Dagger,Martial,7 sp,1d6 P,L,1,Knife,"Agile, Disarm, Orc, Uncommon"
Pick,Martial,7 sp,1d6 P,1,1,Pick,Fatal d10
Ranseur,Martial,2 gp,1d10 P,2,2,Polearm,"Disarm, Reach"
Rapier,Martial,2 gp,1d6 P,1,1,Sword,"Deadly d8, Disarm, Finesse"
Sai,Martial,6 sp,1d4 P,L,1,Knife,"Agile, Disarm, Finesse, Monk, Uncommon, Versatile B"
Sap,Martial,1 sp,1d6 B,L,1,Club,"Agile, Nonlethal"
Scimitar,Martial,1 gp,1d6 S,1,1,Sword,"Forceful, Sweep"
Scythe,Martial,2 gp,1d10 S,2,2,Polearm,"Deadly d10, Trip"
Shield Bash,Martial,,1d4 B,,1,Shield,
Shield Boss,Martial,5 sp,1d6 B,,1,Shield,Attached to Shield
Shield Spikes,Martial,5 sp,1d6 P,,1,Shield,Attached to Shield
Shortsword,Martial,9 sp,1d6 P,L,1,Sword,"Agile, Finesse, Versatile S"
Spiked Chain,Martial,3 gp,1d8 S,1,2,Flail,"Disarm, Finesse, Trip, Uncommon"
Starknife,Martial,2 gp,1d4 P,L,1,Knife,"Agile, Deadly d6, Finesse, Thrown 20 ft., Versatile S"
Temple Sword,Martial,2 gp,1d8 S,1,1,Sword,"Monk, Trip, Uncommon"
Trident,Martial,1 gp,1d8 P,1,1,Spear,Thrown 20 ft.
War Flair,Martial,2 gp,1d10 B,2,2,Flail,"Disarm, Sweep, Trip"
Warhammer,Martial,1 gp,1d8 B,1,1,Hammer,Shove
Whip,Martial,1 sp,1d4 S,1,1,Flail,"Disarm, Finesse, Nonlethal, Reach, Trip"
Aklys,Advanced,5 gp,1d6 B,1,1,Club,"Ranged Trip, Tethered, Thrown 20 feet, Trip, Uncommon"
Dwarven War Axe,Advanced,3 gp,1d8 S,2,1,Axe,"Dwarf, Sweep, Two-Hand d12"
Gnome Flickmace,Advanced,3 gp,1d8 B,2,1,Flail,"Gnome, Reach, Uncommon"
Orc Necksplitter,Advanced,2 gp,1d8 S,1,1,Axe,"Forceful, Orc, Sweep, Uncommon"
Sawtooth Saber,Advanced,5 gp,1d8 S,L,1,Sword,"Agile, Finesse, Twin, Uncommon"
1 Name Category Price Damage Bulk Hands Group Weapon Traits
2 Fist Unarmed 1d4 B 1 Brawling Agile, Finesse, Nonlethal, Unarmed
3 Clan Dagger Simple 2 gp 1d4 P L 1 Knife Agile, Dwarf, Parry, Uncommon, Versatile B
4 Club Simple 0 1d6 B 1 1 Club Thrown 10 ft.
5 Dagger Simple 2 sp 1d4 P L 1 Knife Agile, Finesse, Thrown 10 ft., Versatile S
6 Gauntlet Simple 2 sp 1d4 B L 1 Brawling Agile, Free-Hand
7 Katar Simple 3 sp 1d4 P L 1 Knife Agile, Deadly d6, Monk, Uncommon
8 Light Mace Simple 4 sp 1d4 B L 1 Club Agile, Finesse, Shove
9 Longspear Simple 5 sp 1d8 P 2 2 Spear Reach
10 Mace Simple 1 gp 1d6 B 1 1 Club Shove
11 Morningstar Simple 1 gp 1d6 B 1 1 Club Versatile P
12 Sickle Simple 2 sp 1d4 S L 1 Knife Agile, Finesse, Trip
13 Spear Simple 1 sp 1d6 P 1 1 Spear Thrown 20 ft.
14 Spiked Gauntlet Simple 3 sp 1d4 P L 1 Brawling Agile, Free-Hand
15 Staff Simple 0 1d4 B 1 1 Club Two-Hand d8
16 Bastard Sword Martial 4 gp 1d8 S 1 1 Sword Two-Hand d12
17 Battle Axe Martial 1 sp 1d8 S 1 1 Axe Sweep
18 Bo Staff Martial 2 sp 1d8 B 2 2 Club Monk, Parry, Reach, Trip
19 Dogslicer Martial 1 sp 1d6 S L 1 Sword Agile, Backstabber, Finesse, Goblin, Uncommon
20 Elven Curve Blade Martial 4 gp 1d8 S 2 2 Sword Elf, Finesse, Forceful
21 Falchion Martial 3 gp 1d10 S 2 2 Sword Forceful, Sweep
22 Filcher's Fork Martial 1 gp 1d4 P L 1 Spear Agile, Backstabber, Deadly d6, Finesse, Halfling, Thrown 20 ft., Uncommon
23 Flail Martial 1 gp 1d6 B 1 1 Flail Disarm, Sweep, Trip
24 Glaive Martial 2 gp 1d8 S 2 2 Polearm Deadly d8, Forceful, Reach
25 Gnome Hooked Hammer Martial 2 gp 1d6 B 1 1 Hammer Gnome, Trip, Two-Hand d10, Uncommon, Versatile P
26 Greataxe Martial 1 gp 1d12 S 2 2 Axe Sweep
27 Greatclub Martial 1 gp 1d10 B 2 2 Club Backswing, Shove
28 Greatpick Martial 2 gp 1d10 P 2 2 Pick Fatal d12
29 Greatsword Martial 2 gp 1d12 S 2 2 Sword Versatile P
30 Guisarme Martial 2 gp 1d10 S 2 2 Polearm Reach, Trip
31 Halberd Martial 2 gp 1d10 P 2 2 Polearm Reach, Versatile S
32 Hatchet Martial 4 sp 1d6 S L 1 Axe Agile, Sweep, Thrown 10 ft.
33 Horsechopper Martial 9 sp 1d8 S 2 2 Polearm Goblin, Reach, Trip, Uncommon, Versatile P
34 Kama Martial 1 gp 1d6 S L 1 Knife Agile, Monk, Trip, Uncommon
35 Katana Martial 2 gp 1d6 S 1 1 Sword Deadly d8, Two-Hand d10, Uncommon, Versatile P
36 Kukri Martial 6 sp 1d6 S L 1 Knife Agile, Finesse, Trip, Uncommon
37 Lance Martial 1 gp 1d8 P 2 2 Spear Deadly d8, Jousting d6, Reach
38 Light Hammer Martial 3 sp 1d6 B L 1 Hammer Agile, Thrown 20 ft.
39 Light Pick Martial 4 sp 1d4 P L 1 Pick Agile, Fatal d8
40 Longsword Martial 1 gp 1d8 S 1 1 Sword Versatile P
41 Main-gauche Martial 5 sp 1d4 P L 1 Knife Agile, Disarm, Finesse, Parry, Versatile S
42 Maul Martial 3 gp 1d12 B 2 2 Hammer Shove
43 Nunchaku Martial 2 sp 1d6 B L 1 Club Backswing, Disarm, Finesse, Monk, Uncommon
44 Orc Knuckle Dagger Martial 7 sp 1d6 P L 1 Knife Agile, Disarm, Orc, Uncommon
45 Pick Martial 7 sp 1d6 P 1 1 Pick Fatal d10
46 Ranseur Martial 2 gp 1d10 P 2 2 Polearm Disarm, Reach
47 Rapier Martial 2 gp 1d6 P 1 1 Sword Deadly d8, Disarm, Finesse
48 Sai Martial 6 sp 1d4 P L 1 Knife Agile, Disarm, Finesse, Monk, Uncommon, Versatile B
49 Sap Martial 1 sp 1d6 B L 1 Club Agile, Nonlethal
50 Scimitar Martial 1 gp 1d6 S 1 1 Sword Forceful, Sweep
51 Scythe Martial 2 gp 1d10 S 2 2 Polearm Deadly d10, Trip
52 Shield Bash Martial 1d4 B 1 Shield
53 Shield Boss Martial 5 sp 1d6 B 1 Shield Attached to Shield
54 Shield Spikes Martial 5 sp 1d6 P 1 Shield Attached to Shield
55 Shortsword Martial 9 sp 1d6 P L 1 Sword Agile, Finesse, Versatile S
56 Spiked Chain Martial 3 gp 1d8 S 1 2 Flail Disarm, Finesse, Trip, Uncommon
57 Starknife Martial 2 gp 1d4 P L 1 Knife Agile, Deadly d6, Finesse, Thrown 20 ft., Versatile S
58 Temple Sword Martial 2 gp 1d8 S 1 1 Sword Monk, Trip, Uncommon
59 Trident Martial 1 gp 1d8 P 1 1 Spear Thrown 20 ft.
60 War Flair Martial 2 gp 1d10 B 2 2 Flail Disarm, Sweep, Trip
61 Warhammer Martial 1 gp 1d8 B 1 1 Hammer Shove
62 Whip Martial 1 sp 1d4 S 1 1 Flail Disarm, Finesse, Nonlethal, Reach, Trip
63 Aklys Advanced 5 gp 1d6 B 1 1 Club Ranged Trip, Tethered, Thrown 20 feet, Trip, Uncommon
64 Dwarven War Axe Advanced 3 gp 1d8 S 2 1 Axe Dwarf, Sweep, Two-Hand d12
65 Gnome Flickmace Advanced 3 gp 1d8 B 2 1 Flail Gnome, Reach, Uncommon
66 Orc Necksplitter Advanced 2 gp 1d8 S 1 1 Axe Forceful, Orc, Sweep, Uncommon
67 Sawtooth Saber Advanced 5 gp 1d8 S L 1 Sword Agile, Finesse, Twin, Uncommon

19
src/weapons/ranged.csv 100644
View File

@ -0,0 +1,19 @@
Name,Category,Price,Damage,Range,Reload,Bulk,Hands,Group,Weapon Traits
Blowgun,Simple,1 sp,1.00,20 ft.,1,L,1,Dart,"Agile, Nonlethal"
Crossbow,Simple,3 gp,1d8 P,120 ft.,1,1,2,Bow,
Dart,Simple,1 cp,1d4 P,20 ft.,,L,1,Dart,"Agile, Thrown"
Hand Crossbow,Simple,3 gp,1d6 P,60 ft.,1,L,1,Bow,
Heavy Crossbow,Simple,4 gp,1d10 P,120 ft.,2,2,2,Bow,
Javelin,Simple,1 sp,1d6 P,30 ft.,,L,1,Dart,Thrown
Sling,Simple,0,1d6 B,50 ft.,1,L,1,Sling,Propulsive
Alchemical Bomb,Martial,,Varies,20 ft.,,L,1,Bomb,
Composite Longbow,Martial,20 gp,1d8 P,100 ft.,0,2,1+,Bow,"Deadly d10, Propulsive, Volley 30 ft."
Composite Shortbow,Martial,14 gp,1d6 P,60 ft.,0,1,1+,Bow,"Deadly d10, Propulsive"
Halfling Sling Staff,Martial,5 gp,1d10 B,80 ft.,1,1,2,Sling,"Halfling, Propulsive, Uncommon"
Longbow,Martial,6 gp,1d8 P,100 ft.,0,2,1+,Bow,"Deadly d10, Volley 30 ft."
Shortbow,Martial,3 gp,1d6 P,60 ft.,0,1,1+,Bow,Deadly d10
Shuriken,Martial,1 cp,1d4 P,20 ft.,0,,1,Dart,"Agile, Monk, Thrown, Uncommon"
Arrows,Ammunition,1 sp (price for 10),,,,L,,Bow,
Blowgun Darts,Ammunition,5 cp (price for 10),,,,L,,Dart,
Bolts,Ammunition,1 sp (price for 10),,,,L,,Bow,
Sling Bullets,Ammunition,1 cp (price for 10),,,,L,,Sling,
1 Name Category Price Damage Range Reload Bulk Hands Group Weapon Traits
2 Blowgun Simple 1 sp 1.00 20 ft. 1 L 1 Dart Agile, Nonlethal
3 Crossbow Simple 3 gp 1d8 P 120 ft. 1 1 2 Bow
4 Dart Simple 1 cp 1d4 P 20 ft. L 1 Dart Agile, Thrown
5 Hand Crossbow Simple 3 gp 1d6 P 60 ft. 1 L 1 Bow
6 Heavy Crossbow Simple 4 gp 1d10 P 120 ft. 2 2 2 Bow
7 Javelin Simple 1 sp 1d6 P 30 ft. L 1 Dart Thrown
8 Sling Simple 0 1d6 B 50 ft. 1 L 1 Sling Propulsive
9 Alchemical Bomb Martial Varies 20 ft. L 1 Bomb
10 Composite Longbow Martial 20 gp 1d8 P 100 ft. 0 2 1+ Bow Deadly d10, Propulsive, Volley 30 ft.
11 Composite Shortbow Martial 14 gp 1d6 P 60 ft. 0 1 1+ Bow Deadly d10, Propulsive
12 Halfling Sling Staff Martial 5 gp 1d10 B 80 ft. 1 1 2 Sling Halfling, Propulsive, Uncommon
13 Longbow Martial 6 gp 1d8 P 100 ft. 0 2 1+ Bow Deadly d10, Volley 30 ft.
14 Shortbow Martial 3 gp 1d6 P 60 ft. 0 1 1+ Bow Deadly d10
15 Shuriken Martial 1 cp 1d4 P 20 ft. 0 1 Dart Agile, Monk, Thrown, Uncommon
16 Arrows Ammunition 1 sp (price for 10) L Bow
17 Blowgun Darts Ammunition 5 cp (price for 10) L Dart
18 Bolts Ammunition 1 sp (price for 10) L Bow
19 Sling Bullets Ammunition 1 cp (price for 10) L Sling

153
src/weapons/scrape.ipynb 100644
View File

@ -0,0 +1,153 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Scrape data from aon2e and generate csvs to import in to sqlite"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"# Dependencies\n",
"import pandas as pd\n",
"from bs4 import BeautifulSoup as bs\n",
"import time\n",
"import re\n",
"import os\n",
"from splinter import Browser\n",
"\n",
"# Setting up Selenium\n",
"chrome_driver = os.path.join('..', 'resources', 'chromedriver.exe')\n",
"executable_path = {'executable_path': chrome_driver}\n",
"browser = Browser('chrome', **executable_path, headless=False)\n",
"\n",
"# Pandas config\n",
"pd.set_option('display.max_columns', None)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# url that contains all the links\n",
"url_weapon = 'https://2e.aonprd.com/Weapons.aspx?ID='\n",
"\n",
"# Number of weapons\n",
"number_of_weapons = 83\n",
"\n",
"# Empty lists to store the scraped values\n",
"name_list = []\n",
"description_list = []\n",
"\n",
"print(f'Beginning Data Retrieval')\n",
"print(f'------------------------')\n",
"\n",
"# Loop from 1 to the value in weapon_number\n",
"for weapon in range(1, weapon_number):\n",
" \n",
" try:\n",
" # Make the request to the aon2e\n",
" response_weapon = requests.get(f'{url_weapon}{weapon}')\n",
"\n",
" # Use BS4 html parser to generate soup\n",
" soup_weapon = bs(response_weapon.text, 'html.parser')\n",
"\n",
" # Select only the content section\n",
" content = soup_weapon.find(id='ctl00_MainContent_DetailedOutput')\n",
"\n",
" # Store the name and description\n",
" name = content.find('a').text.strip()\n",
" name_list.append(name)\n",
"\n",
" print(f'Processing Weapon {weapon} of Set {weapon_number+1} | {name}')\n",
" \n",
" description = content.find('hr').next.strip()\n",
" \n",
" # Append values to our empty lists \n",
" description_list.append(description)\n",
" \n",
" # Sleep for 1s to play nice :D\n",
" time.sleep(1.01)\n",
" \n",
" except:\n",
" description = 'error skipping'\n",
" print(description) \n",
"\n",
"print(f'------------------------')\n",
"print(f'Data Retrieval Complete')"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Nethys Note: no description was provided for this item\n"
]
}
],
"source": [
"# url that contains all the links\n",
"url_weapon = 'https://2e.aonprd.com/Weapons.aspx?ID='\n",
"\n",
"# Number of weapons\n",
"weapon_number = 1\n",
"\n",
"url = url_weapon + str(weapon_number)\n",
"browser.visit(url)\n",
"\n",
"html = browser.html\n",
"soup = bs(html, 'html.parser')\n",
"\n",
"content = soup.find(id=\"ctl00_MainContent_DetailedOutput\").find('hr').next.text\n",
"\n",
"print(content)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Directory of csv files which are taken from https://2e.aonprd.com/Weapons.aspx\n",
"melee_csv = os.path.join('weapons', 'melee.csv')\n",
"ranged_csv = os.path.join('weapons', 'ranged.csv')\n",
"melee = pd.read_csv(melee_csv)\n",
"ranged = pd.read_csv(ranged_csv)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
}
},
"nbformat": 4,
"nbformat_minor": 4
}