Wikidata

Wikidata is one of the newer families added to the wikimedia projects. It acts as a central storage for structured data for all the wikimedia projects. It solves 2 major problems that wikimedia projects used to face:

  • Copied information (If the information changes it has to be manually edited in each website)
  • Unstructured data (A better method of storing data which can be queried using SQL or similar)
import pywikibot
wikidata = pywikibot.Site('wikidata', 'wikidata')
wikidata
DataSite("wikidata", "wikidata")
testwikidata = pywikibot.Site('test', 'wikidata')
testwikidata
DataSite("test", "wikidata")

1. Items and Properties

In wikidata, every page is either an item or a property.

Items are used to represent all the things in human knowledge: including topics, concepts, and objects. For example; color (Q1075), Albert Einstein (Q937), Earth (Q2), and cat (Q146) are all considered as items in Wikidata.

Properties are the things that describe and define a item. Each data bit related to an item is a type of property. Properties are different for different types of items. Examples of properties for Python (Q28865) are: license (P275), bug tracking system (P1401), official website (P856), Stack exchange tag (P1482).

The wikidata API helps to query data form wikidata using SparkQL to filter properties. So, for example you can find all countries in the world which have a population between 10 million to 300 million with just 1 query. In the earlier category interface, there would have to be a category for this information or you would have to parse evry country's page to find the population using Natural language processing!

Exercise - Contribute to wikidata

Find an item on wikidata and edit it to add some additional information. Some tips on finding an item:

  • Your favourite book or author
  • Your city or state, or a popular place near your hometown
  • A software, tool, or programming language that you like using
  • If you can type in a native language, try translating the label/title of an item

2. Examples of bots with wikidata

The wikidata game

The wikidata game is an example of a bot which helps users contribute better to wikidata. You can check out the wikidata game at https://tools.wmflabs.org/wikidata-game

The wikidata game finds possible pages which do not have a certain type of information using the structured queries (For example human items which have no gender) and shows the wikipedia page related to that item. Then the user is expected to identify a specific property of the item (For example male or female for the property gender).

The wikidata resonator

Wikidata resonator is a script which pulls data from wikidata and joins all the property data of an item to form a descriptive paragraph about the item. You can check it out at https://tools.wmflabs.org/reasonator/

Other than forming a descriptive paragraph of the item, it also groups similar properites like "Relative" group, "External sources" etc. based on some simple conditions. It also creates a timeline of the item if possible based on the properties that would have a datetime data type. It also generates a QR Code to use for the related wikipedia page and shows related images pulled from the related commons.wikimedia page!

3. Fetching data from wikidata using pywikibot

The first thing we're going to do is figure out how to get data from wikidata using pywikibot. Here, it's not like a generic mediawiki website, where the text of the page is pulled. Here, the data is structured. We use a ItemPage class in pywikibot which can handle these items in a better way:

itempage = pywikibot.ItemPage(wikidata, "Q42")  # Q42 is Douglas Adams
itempage
ItemPage('Q42')

In wikidata, the page.text won't work like other mediawiki websites where it gives a string of the whole content in the page. The data and properties are stored in Python dictionary structure:

itempage.get()
{'aliases': {'be-tarask': ['Дуглас Адамс'],
  'cs': ['Douglas Noël Adams', 'Douglas Noel Adams'],
  'de': ['Douglas Noël Adams', 'Douglas Noel Adams'],
  'el': ['Ντάγκλας Νόελ Άνταμς'],
  'en': ['Douglas Noël Adams', 'Douglas Noel Adams'],
  'es': ['Douglas Noel Adams', 'Douglas Noël Adams'],
  'fr': ['Douglas Noel Adams', 'Douglas Noël Adams'],
  'hy': ['Ադամս, Դուգլաս'],
  'it': ['Douglas Noel Adams'],
  'ja': ['ダグラス・アダムス'],
  'nb': ['Douglas Noël Adams'],
  'nl': ['Douglas Noel Adams', 'Douglas Noël Adams'],
  'pa': ['ਡਗਲਸ ਨੋਏਲ ਐਡਮਜ਼'],
  'pt': ['Douglas Noël Adams', 'Douglas Noel Adams'],
  'pt-br': ['Douglas Noël Adams', 'Douglas Noel Adams'],
  'ru': ['Адамс, Дуглас'],
  'zh': ['亞當斯']},
 'claims': {'P1003': [<pywikibot.page.Claim at 0x7f383c8c2d68>],
  'P1005': [<pywikibot.page.Claim at 0x7f383c928d68>],
  'P1006': [<pywikibot.page.Claim at 0x7f383c916080>,
   <pywikibot.page.Claim at 0x7f383c916240>],
  'P1015': [<pywikibot.page.Claim at 0x7f383c930198>],
  'P103': [<pywikibot.page.Claim at 0x7f383c8bde48>,
   <pywikibot.page.Claim at 0x7f383c8c22e8>],
  'P106': [<pywikibot.page.Claim at 0x7f383c8f0cf8>,
   <pywikibot.page.Claim at 0x7f383c8f0e48>,
   <pywikibot.page.Claim at 0x7f383c8f0f98>,
   <pywikibot.page.Claim at 0x7f383c8f4128>,
   <pywikibot.page.Claim at 0x7f383c8f4588>,
   <pywikibot.page.Claim at 0x7f383c8f46a0>,
   <pywikibot.page.Claim at 0x7f383c8f4748>,
   <pywikibot.page.Claim at 0x7f383c8f4a20>],
  'P108': [<pywikibot.page.Claim at 0x7f383c90bfd0>],
  'P119': [<pywikibot.page.Claim at 0x7f383c8cc278>],
  'P1196': [<pywikibot.page.Claim at 0x7f383c8c6fd0>],
  'P1207': [<pywikibot.page.Claim at 0x7f383c8e69e8>],
  'P1233': [<pywikibot.page.Claim at 0x7f383c8cc240>],
  'P1258': [<pywikibot.page.Claim at 0x7f383c8c2e10>],
  'P1263': [<pywikibot.page.Claim at 0x7f383c8bdda0>],
  'P1266': [<pywikibot.page.Claim at 0x7f383c928ef0>],
  'P1273': [<pywikibot.page.Claim at 0x7f383c8d2c50>],
  'P1284': [<pywikibot.page.Claim at 0x7f383c8f0a20>],
  'P1315': [<pywikibot.page.Claim at 0x7f383c8c2940>],
  'P136': [<pywikibot.page.Claim at 0x7f383c8c2f28>],
  'P1368': [<pywikibot.page.Claim at 0x7f383c930c18>],
  'P1375': [<pywikibot.page.Claim at 0x7f383c928e80>],
  'P140': [<pywikibot.page.Claim at 0x7f383cfd3550>],
  'P1411': [<pywikibot.page.Claim at 0x7f383c8cce80>,
   <pywikibot.page.Claim at 0x7f383c8ccf98>],
  'P1412': [<pywikibot.page.Claim at 0x7f383c8bd048>],
  'P1415': [<pywikibot.page.Claim at 0x7f383c92c278>],
  'P1417': [<pywikibot.page.Claim at 0x7f383c8f0860>],
  'P1442': [<pywikibot.page.Claim at 0x7f383c8d2e48>],
  'P1477': [<pywikibot.page.Claim at 0x7f383c930780>],
  'P1559': [<pywikibot.page.Claim at 0x7f383c8f0ba8>],
  'P1670': [<pywikibot.page.Claim at 0x7f383c925eb8>],
  'P1695': [<pywikibot.page.Claim at 0x7f383c930710>],
  'P172': [<pywikibot.page.Claim at 0x7f383c928c18>],
  'P18': [<pywikibot.page.Claim at 0x7f383c9164e0>],
  'P1816': [<pywikibot.page.Claim at 0x7f383c92c2e8>],
  'P19': [<pywikibot.page.Claim at 0x7f383c8c2e80>],
  'P1953': [<pywikibot.page.Claim at 0x7f383c92c208>],
  'P20': [<pywikibot.page.Claim at 0x7f383c8e6a58>],
  'P2019': [<pywikibot.page.Claim at 0x7f383c8bd748>],
  'P21': [<pywikibot.page.Claim at 0x7f383c8d2208>],
  'P213': [<pywikibot.page.Claim at 0x7f383c910a20>],
  'P214': [<pywikibot.page.Claim at 0x7f383c8f4da0>],
  'P2163': [<pywikibot.page.Claim at 0x7f383c8f4ba8>],
  'P2168': [<pywikibot.page.Claim at 0x7f383c92c048>],
  'P2188': [<pywikibot.page.Claim at 0x7f383c8c26a0>],
  'P2191': [<pywikibot.page.Claim at 0x7f383c930208>],
  'P22': [<pywikibot.page.Claim at 0x7f383c92c5f8>],
  'P227': [<pywikibot.page.Claim at 0x7f383c8d6048>],
  'P2387': [<pywikibot.page.Claim at 0x7f383c8dcf28>],
  'P2435': [<pywikibot.page.Claim at 0x7f383c8f4e80>],
  'P244': [<pywikibot.page.Claim at 0x7f383c910860>],
  'P25': [<pywikibot.page.Claim at 0x7f383c8dcac8>],
  'P26': [<pywikibot.page.Claim at 0x7f383c8e6278>],
  'P2604': [<pywikibot.page.Claim at 0x7f383c92c588>],
  'P2605': [<pywikibot.page.Claim at 0x7f383c8e6f28>],
  'P2611': [<pywikibot.page.Claim at 0x7f383c8cc588>],
  'P2626': [<pywikibot.page.Claim at 0x7f383c930ba8>],
  'P268': [<pywikibot.page.Claim at 0x7f383c910ba8>],
  'P269': [<pywikibot.page.Claim at 0x7f383c8d2518>],
  'P27': [<pywikibot.page.Claim at 0x7f383c935518>],
  'P271': [<pywikibot.page.Claim at 0x7f383c8f4cc0>],
  'P2963': [<pywikibot.page.Claim at 0x7f383c8d2550>],
  'P31': [<pywikibot.page.Claim at 0x7f383c8bd400>],
  'P345': [<pywikibot.page.Claim at 0x7f383c8cc3c8>],
  'P349': [<pywikibot.page.Claim at 0x7f383c8f0b38>],
  'P373': [<pywikibot.page.Claim at 0x7f383c8ccc88>],
  'P396': [<pywikibot.page.Claim at 0x7f383c8e6d68>],
  'P40': [<pywikibot.page.Claim at 0x7f383c8bd898>],
  'P409': [<pywikibot.page.Claim at 0x7f383c930668>],
  'P434': [<pywikibot.page.Claim at 0x7f383c8f4e10>],
  'P509': [<pywikibot.page.Claim at 0x7f383c8d61d0>],
  'P535': [<pywikibot.page.Claim at 0x7f383c8dceb8>],
  'P551': [<pywikibot.page.Claim at 0x7f383c8eb828>,
   <pywikibot.page.Claim at 0x7f383c8eba20>,
   <pywikibot.page.Claim at 0x7f383c8ebbe0>],
  'P569': [<pywikibot.page.Claim at 0x7f383c935358>],
  'P570': [<pywikibot.page.Claim at 0x7f383c92c518>],
  'P646': [<pywikibot.page.Claim at 0x7f383c928278>],
  'P648': [<pywikibot.page.Claim at 0x7f383c8c28d0>],
  'P69': [<pywikibot.page.Claim at 0x7f383c8e23c8>,
   <pywikibot.page.Claim at 0x7f383c8e27f0>],
  'P691': [<pywikibot.page.Claim at 0x7f383c8e6438>],
  'P734': [<pywikibot.page.Claim at 0x7f383c910da0>],
  'P735': [<pywikibot.page.Claim at 0x7f383c8d6470>,
   <pywikibot.page.Claim at 0x7f383c8dc710>],
  'P800': [<pywikibot.page.Claim at 0x7f383c9282e8>,
   <pywikibot.page.Claim at 0x7f383c928630>,
   <pywikibot.page.Claim at 0x7f383c9288d0>,
   <pywikibot.page.Claim at 0x7f383c928a58>,
   <pywikibot.page.Claim at 0x7f383c928ac8>],
  'P856': [<pywikibot.page.Claim at 0x7f383c8d60b8>],
  'P866': [<pywikibot.page.Claim at 0x7f383c8f4f98>],
  'P9': [<pywikibot.page.Claim at 0x7f383c8c27b8>],
  'P906': [<pywikibot.page.Claim at 0x7f383c8d2f98>],
  'P910': [<pywikibot.page.Claim at 0x7f383c8ebd68>],
  'P947': [<pywikibot.page.Claim at 0x7f383c8c6eb8>],
  'P949': [<pywikibot.page.Claim at 0x7f383c9352b0>],
  'P950': [<pywikibot.page.Claim at 0x7f383c910a90>],
  'P998': [<pywikibot.page.Claim at 0x7f383c8ebf98>,
   <pywikibot.page.Claim at 0x7f383c8ebfd0>,
   <pywikibot.page.Claim at 0x7f383c8f0160>,
   <pywikibot.page.Claim at 0x7f383c8f0320>,
   <pywikibot.page.Claim at 0x7f383c8f04e0>,
   <pywikibot.page.Claim at 0x7f383c8f06a0>]},
 'descriptions': {'af': 'Engelse skrywer en humoris',
  'ar': 'كاتب إنجليزي فكاهي',
  'bar': 'a englischer Science-Fiction-Schriftsteller',
  'bg': 'английски писател и хуморист',
  'br': 'skrivagner saoznek',
  'ca': 'escriptor anglès',
  'cs': 'anglický spisovatel, humorista a dramatik',
  'cy': 'awdur a dychanwr Seisnig',
  'da': 'engelsk forfatter',
  'de': 'britischer Schriftsteller',
  'de-at': 'britischer Schriftsteller',
  'de-ch': 'britischer Schriftsteller',
  'el': 'Άγγλος συγγραφέας',
  'en': 'English writer and humorist',
  'en-gb': 'English writer and humourist',
  'eo': 'angla aŭtoro de sciencfikcio-romanoj kaj humoristo',
  'es': 'escritor y humorista británico',
  'fa': 'فیلمنامه\u200cنویس و نویسنده بریتانیایی',
  'fi': 'englantilainen kirjailija ja humoristi',
  'fr': 'écrivain anglais de science-fiction',
  'gsw': 'britischer Schriftsteller',
  'gu': 'અંગ્રેજી લેખક અને હાસ્યકાર',
  'he': 'סופר והומוריסטן בריטי',
  'hu': 'angol író',
  'hy': 'անգլիացի գրող, դրամատուրգ, սցենարիստ, «Ավտոստոպով զբոսաշրջիկի միջգալակտիկական ուղեցույց» վեպերի շարք',
  'it': 'scrittore inglese',
  'ja': 'イングランドの作家',
  'kn': 'ಇಂಗ್ಲಿಷ್ ಭಾಷೆಯ ಬರಹಗಾರ ಹಾಗೂ ಹಾಸ್ಯ ಲೇಖಕ',
  'lv': 'angļu zinātniskās fantastikas rakstnieks un humorists',
  'mk': 'англиски писател и хуморист',
  'nb': 'engelsk science fiction-forfatter og humorist',
  'nds': 'englischer Schriftsteller',
  'ne': 'अङ्ग्रेजी लेखक र व्यङ्ग्यकार',
  'nl': 'Engelse schrijver',
  'nn': 'engelsk sciencefictionforfattar og humorist',
  'pa': 'ਅੰਗਰੇਜ਼ੀ ਲੇਖਕ',
  'pl': 'brytyjski pisarz',
  'pt': 'escritor e comediante britânico',
  'pt-br': 'escritor e humorista inglês',
  'ro': 'scriitor, dramaturg englez',
  'ru': 'английский писатель, драматург и сценарист, автор серии книг «Автостопом по галактике»',
  'si': 'ඉංග්\u200dරීසි කවියෙක්',
  'sl': 'angleški pisatelj, humorist in dramatik',
  'sr': 'енглески писац научне фантастике и хумориста',
  'sr-ec': 'енглески писац научне фантастике и хумориста',
  'sr-el': 'engleski pisac naučne fantastike i humorista',
  'sv': 'brittisk författare och humorist',
  'ta': 'ஆங்கில எழுத்தாளர் மற்றும் நகைச்சுவையாளர்',
  'tr': 'İngiliz bilim kurgu ve mizah yazarı',
  'uk': 'британський комічний радіодраматург, письменник',
  'vi': 'Nhà văn và nhà soạn hài kịch người Anh',
  'zh': '英国作家',
  'zh-cn': '英国作家',
  'zh-hans': '英国作家',
  'zh-hant': '英國作家',
  'zh-hk': '英國作家',
  'zh-mo': '英國作家',
  'zh-my': '英国作家',
  'zh-sg': '英国作家',
  'zh-tw': '英國作家'},
 'labels': {'af': 'Douglas Adams',
  'ak': 'Doglas Adams',
  'an': 'Douglas Adams',
  'ar': 'دوغلاس آدمز',
  'arz': 'دوجلاس ادامز',
  'ast': 'Douglas Adams',
  'az': 'Duqlas Noel Adams',
  'bar': 'Douglas Adams',
  'be': 'Дуглас Адамс',
  'be-tarask': 'Дуглас Адамз',
  'be-x-old': 'Дуглас Адамс',
  'bg': 'Дъглас Адамс',
  'bn': 'ডগলাস অ্যাডামস',
  'br': 'Douglas Adams',
  'bs': 'Douglas Adams',
  'ca': 'Douglas Adams',
  'ckb': 'دەگلاس ئادمز',
  'co': 'Douglas Adams',
  'cs': 'Douglas Adams',
  'cy': 'Douglas Adams',
  'da': 'Douglas Adams',
  'de': 'Douglas Adams',
  'de-at': 'Douglas Adams',
  'de-ch': 'Douglas Adams',
  'el': 'Ντάγκλας Άνταμς',
  'en': 'Douglas Adams',
  'en-ca': 'Douglas Adams',
  'en-gb': 'Douglas Adams',
  'eo': 'Douglas Adams',
  'es': 'Douglas Adams',
  'et': 'Douglas Adams',
  'eu': 'Douglas Adams',
  'fa': 'داگلاس آدامز',
  'fi': 'Douglas Adams',
  'fo': 'Douglas Adams',
  'fr': 'Douglas Adams',
  'frp': 'Douglas Adams',
  'fur': 'Douglas Adams',
  'ga': 'Douglas Adams',
  'gd': 'Douglas Adams',
  'gl': 'Douglas Adams',
  'gsw': 'Douglas Adams',
  'gu': 'ડગ્લાસ એડમ્સ',
  'he': 'דאגלס אדאמס',
  'hi': 'डग्लस अ\u200dडम्स',
  'hr': 'Douglas Adams',
  'hu': 'Douglas Adams',
  'hy': 'Դուգլաս Ադամս',
  'ia': 'Douglas Adams',
  'id': 'Douglas Adams',
  'ie': 'Douglas Adams',
  'io': 'Douglas Adams',
  'is': 'Douglas Adams',
  'it': 'Douglas Adams',
  'ja': 'ダグラス・アダムズ',
  'jv': 'Douglas Adams',
  'ka': 'დაგლას ადამსი',
  'kg': 'Douglas Adams',
  'kl': 'Douglas Adams',
  'kn': 'ಡಾಗ್ಲಸ್ ಆಡಮ್ಸ್',
  'ko': '더글러스 애덤스',
  'la': 'Duglassius Adams',
  'lb': 'Douglas Adams',
  'li': 'Douglas Adams',
  'lij': 'Douglas Adams',
  'lmo': 'Douglas Adams',
  'lt': 'Douglas Adams',
  'lv': 'Duglass Adamss',
  'mg': 'Douglas Adams',
  'min': 'Douglas Adams',
  'mk': 'Даглас Адамс',
  'ml': 'ഡഗ്ലസ് ആഡംസ്',
  'mr': 'डग्लस अॅडम्स',
  'mrj': 'Адамс',
  'ms': 'Douglas Adams',
  'nap': 'Douglas Adams',
  'nb': 'Douglas Adams',
  'nds': 'Douglas Adams',
  'nds-nl': 'Douglas Adams',
  'ne': 'डगलस एडम्स',
  'nl': 'Douglas Adams',
  'nn': 'Douglas Adams',
  'nrm': 'Douglas Adams',
  'oc': 'Douglas Adams',
  'or': 'ଡଗ୍\u200cଲାସ୍\u200c ଆଦାମ୍\u200cସ',
  'pa': 'ਡਗਲਸ ਐਡਮਜ਼',
  'pcd': 'Douglas Adams',
  'pl': 'Douglas Adams',
  'pms': 'Douglas Adams',
  'pt': 'Douglas Adams',
  'pt-br': 'Douglas Adams',
  'rm': 'Douglas Adams',
  'ro': 'Douglas Adams',
  'ru': 'Дуглас Адамс',
  'rwr': 'डग्लस अ\u200dडम्स',
  'sc': 'Douglas Adams',
  'scn': 'Douglas Adams',
  'sco': 'Douglas Adams',
  'sh': 'Douglas Adams',
  'si': 'ඩග්ලස් ඇඩම්ස්',
  'sk': 'Douglas Adams',
  'sl': 'Douglas Adams',
  'sq': 'Douglas Adams',
  'sr': 'Даглас Адамс',
  'sr-ec': 'Даглас Адамс',
  'sr-el': 'Daglas Adams',
  'sv': 'Douglas Adams',
  'sw': 'Douglas Adams',
  'ta': 'டக்ளஸ் ஆடம்ஸ்',
  'tcy': 'ಡಾಗ್ಲಸ್ ಆಡಮ್ಸ್',
  'te': 'డగ్లస్ ఆడమ్స్',
  'th': 'ดั๊กลาส อดัมส์',
  'tr': 'Douglas Adams',
  'uk': 'Дуглас Адамс',
  'ur': 'ڈگلس ایڈم',
  'vec': 'Douglas Adams',
  'vep': 'Adams Duglas',
  'vi': 'Douglas Adams',
  'vls': 'Douglas Adams',
  'vo': 'Douglas Adams',
  'wa': 'Douglas Adams',
  'war': 'Douglas Adams',
  'wo': 'Douglas Adams',
  'zh': '道格拉斯·亚当斯',
  'zh-cn': '道格拉斯·亚当斯',
  'zh-hans': '道格拉斯·亚当斯',
  'zh-hant': '道格拉斯·亞當斯',
  'zh-hk': '道格拉斯·亞當斯',
  'zh-mo': '道格拉斯·亞當斯',
  'zh-my': '道格拉斯·亚当斯',
  'zh-sg': '道格拉斯·亚当斯',
  'zh-tw': '道格拉斯·亞當斯',
  'zu': 'Douglas Adams'},
 'sitelinks': {'arwiki': 'دوغلاس آدمز',
  'arzwiki': 'دوجلاس ادامز',
  'astwiki': 'Douglas Adams',
  'azwiki': 'Duqlas Adams',
  'azwikiquote': 'Duqlas Noel Adams',
  'barwiki': 'Douglas Adams',
  'be_x_oldwiki': 'Дуглас Адамз',
  'bewiki': 'Дуглас Адамс',
  'bgwiki': 'Дъглас Адамс',
  'bgwikiquote': 'Дъглас Адамс',
  'bnwiki': 'ডগলাস অ্যাডামস',
  'bswiki': 'Douglas Adams',
  'bswikiquote': 'Douglas Adams',
  'cawiki': 'Douglas Adams',
  'cswiki': 'Douglas Adams',
  'cswikiquote': 'Douglas Adams',
  'cywiki': 'Douglas Adams',
  'dawiki': 'Douglas Adams',
  'dewiki': 'Douglas Adams',
  'dewikiquote': 'Douglas Adams',
  'elwiki': 'Ντάγκλας Άνταμς',
  'elwikiquote': 'Ντάγκλας Άνταμς',
  'enwiki': 'Douglas Adams',
  'enwikiquote': 'Douglas Adams',
  'eowiki': 'Douglas Adams',
  'eowikiquote': 'Douglas Adams',
  'eswiki': 'Douglas Adams',
  'eswikiquote': 'Douglas Adams',
  'etwiki': 'Douglas Adams',
  'etwikiquote': 'Douglas Adams',
  'euwiki': 'Douglas Adams',
  'fawiki': 'داگلاس آدامز',
  'fawikiquote': 'داگلاس آدامز',
  'fiwiki': 'Douglas Adams',
  'fiwikiquote': 'Douglas Adams',
  'frwiki': 'Douglas Adams',
  'frwikiquote': 'Douglas Adams',
  'gawiki': 'Douglas Adams',
  'glwiki': 'Douglas Adams',
  'glwikiquote': 'Douglas Adams',
  'hewiki': 'דאגלס אדמס',
  'hewikiquote': 'דאגלס אדמס',
  'hrwiki': 'Douglas Adams',
  'huwiki': 'Douglas Adams',
  'huwikiquote': 'Douglas Adams',
  'hywiki': 'Դուգլաս Ադամս',
  'hywikiquote': 'Դուգլաս Ադամս',
  'idwiki': 'Douglas Adams',
  'iowiki': 'Douglas Adams',
  'iswiki': 'Douglas Adams',
  'itwiki': 'Douglas Adams',
  'itwikiquote': 'Douglas Adams',
  'jawiki': 'ダグラス・アダムズ',
  'jvwiki': 'Douglas Adams',
  'kawiki': 'დაგლას ადამსი',
  'kowiki': '더글러스 애덤스',
  'lawiki': 'Duglassius Adams',
  'liwikiquote': 'Douglas Adams',
  'ltwikiquote': 'Douglas Adamsas',
  'lvwiki': 'Duglass Adamss',
  'mgwiki': 'Douglas Adams',
  'mkwiki': 'Даглас Адамс',
  'mlwiki': 'ഡഗ്ലസ് ആഡംസ്',
  'mrjwiki': 'Адамс, Дуглас',
  'mrwiki': 'डग्लस अ\u200dॅडम्स',
  'nlwiki': 'Douglas Adams',
  'nlwikiquote': 'Douglas Adams',
  'nnwiki': 'Douglas Adams',
  'nowiki': 'Douglas Adams',
  'ocwiki': 'Douglas Adams',
  'plwiki': 'Douglas Adams',
  'plwikiquote': 'Douglas Adams',
  'ptwiki': 'Douglas Adams',
  'ptwikiquote': 'Douglas Adams',
  'rowiki': 'Douglas Adams',
  'ruwiki': 'Адамс, Дуглас',
  'ruwikiquote': 'Дуглас Ноэль Адамс',
  'scowiki': 'Douglas Adams',
  'shwiki': 'Douglas Adams',
  'simplewiki': 'Douglas Adams',
  'simplewikiquote': 'Douglas Adams',
  'skwiki': 'Douglas Adams',
  'skwikiquote': 'Douglas Adams',
  'slwiki': 'Douglas Adams',
  'sqwiki': 'Douglas Adams',
  'srwiki': 'Даглас Адамс',
  'svwiki': 'Douglas Adams',
  'svwikiquote': 'Douglas Adams',
  'tawiki': 'டக்ளஸ் ஆடம்ஸ்',
  'thwikiquote': 'ดั๊กลาส อดัมส์',
  'trwiki': 'Douglas Adams',
  'trwikiquote': 'Douglas Adams',
  'ukwiki': 'Дуглас Адамс',
  'urwiki': 'ڈگلس ایڈمس',
  'vepwiki': 'Adams Duglas',
  'viwiki': 'Douglas Adams',
  'warwiki': 'Douglas Adams',
  'zhwiki': '道格拉斯·亚当斯',
  'zhwikiquote': '道格拉斯·亞當斯'}}

If you want to get the data using the title of the page rather than the item ID, we can get the wikidata article associated to a wikipedia page using:

itempage == pywikibot.ItemPage.fromPage(pywikibot.Page(pywikibot.Site('en', 'wikipedia'), 'Douglas Adams'))
True

Let's take a closer look at the data items given by an item page. It is a dictionary with the following keys:

itemdata = itempage.get()
itemdata.keys()
dict_keys(['aliases', 'claims', 'sitelinks', 'labels', 'descriptions'])

Labels, Descriptions and Aliases

Labels are the name or title of the wikidata item.

Aliases are alternate labels for the same item in the same lanugage. For example, "Python (Q28865)" The programming language has the alias "Python language", "Python programming language", "/usr/bin/python", etc.

Descriptions are useful statements which can help distinguish items with similar labels. Wikidata items are unique only by their item ID (Qxx) hence the description helps differentiate behind "Python (Q271218)" the genus of reptiles, and "Python (Q28865)" the programming language, and "Python (Q15728)" the family of missiles!

As wikidata does not have a specific code/language (the code we use is "wikidata") it has data for all languages. Hence, the same item can have a different label in Engligh, Arabic, or French. So, these fields in the data are dictionaries with the key as the language code and the value as the label in that language.

itemdata['labels']
{'af': 'Douglas Adams',
 'ak': 'Doglas Adams',
 'an': 'Douglas Adams',
 'ar': 'دوغلاس آدمز',
 'arz': 'دوجلاس ادامز',
 'ast': 'Douglas Adams',
 'az': 'Duqlas Noel Adams',
 'bar': 'Douglas Adams',
 'be': 'Дуглас Адамс',
 'be-tarask': 'Дуглас Адамз',
 'be-x-old': 'Дуглас Адамс',
 'bg': 'Дъглас Адамс',
 'bn': 'ডগলাস অ্যাডামস',
 'br': 'Douglas Adams',
 'bs': 'Douglas Adams',
 'ca': 'Douglas Adams',
 'ckb': 'دەگلاس ئادمز',
 'co': 'Douglas Adams',
 'cs': 'Douglas Adams',
 'cy': 'Douglas Adams',
 'da': 'Douglas Adams',
 'de': 'Douglas Adams',
 'de-at': 'Douglas Adams',
 'de-ch': 'Douglas Adams',
 'el': 'Ντάγκλας Άνταμς',
 'en': 'Douglas Adams',
 'en-ca': 'Douglas Adams',
 'en-gb': 'Douglas Adams',
 'eo': 'Douglas Adams',
 'es': 'Douglas Adams',
 'et': 'Douglas Adams',
 'eu': 'Douglas Adams',
 'fa': 'داگلاس آدامز',
 'fi': 'Douglas Adams',
 'fo': 'Douglas Adams',
 'fr': 'Douglas Adams',
 'frp': 'Douglas Adams',
 'fur': 'Douglas Adams',
 'ga': 'Douglas Adams',
 'gd': 'Douglas Adams',
 'gl': 'Douglas Adams',
 'gsw': 'Douglas Adams',
 'gu': 'ડગ્લાસ એડમ્સ',
 'he': 'דאגלס אדאמס',
 'hi': 'डग्लस अ\u200dडम्स',
 'hr': 'Douglas Adams',
 'hu': 'Douglas Adams',
 'hy': 'Դուգլաս Ադամս',
 'ia': 'Douglas Adams',
 'id': 'Douglas Adams',
 'ie': 'Douglas Adams',
 'io': 'Douglas Adams',
 'is': 'Douglas Adams',
 'it': 'Douglas Adams',
 'ja': 'ダグラス・アダムズ',
 'jv': 'Douglas Adams',
 'ka': 'დაგლას ადამსი',
 'kg': 'Douglas Adams',
 'kl': 'Douglas Adams',
 'kn': 'ಡಾಗ್ಲಸ್ ಆಡಮ್ಸ್',
 'ko': '더글러스 애덤스',
 'la': 'Duglassius Adams',
 'lb': 'Douglas Adams',
 'li': 'Douglas Adams',
 'lij': 'Douglas Adams',
 'lmo': 'Douglas Adams',
 'lt': 'Douglas Adams',
 'lv': 'Duglass Adamss',
 'mg': 'Douglas Adams',
 'min': 'Douglas Adams',
 'mk': 'Даглас Адамс',
 'ml': 'ഡഗ്ലസ് ആഡംസ്',
 'mr': 'डग्लस अॅडम्स',
 'mrj': 'Адамс',
 'ms': 'Douglas Adams',
 'nap': 'Douglas Adams',
 'nb': 'Douglas Adams',
 'nds': 'Douglas Adams',
 'nds-nl': 'Douglas Adams',
 'ne': 'डगलस एडम्स',
 'nl': 'Douglas Adams',
 'nn': 'Douglas Adams',
 'nrm': 'Douglas Adams',
 'oc': 'Douglas Adams',
 'or': 'ଡଗ୍\u200cଲାସ୍\u200c ଆଦାମ୍\u200cସ',
 'pa': 'ਡਗਲਸ ਐਡਮਜ਼',
 'pcd': 'Douglas Adams',
 'pl': 'Douglas Adams',
 'pms': 'Douglas Adams',
 'pt': 'Douglas Adams',
 'pt-br': 'Douglas Adams',
 'rm': 'Douglas Adams',
 'ro': 'Douglas Adams',
 'ru': 'Дуглас Адамс',
 'rwr': 'डग्लस अ\u200dडम्स',
 'sc': 'Douglas Adams',
 'scn': 'Douglas Adams',
 'sco': 'Douglas Adams',
 'sh': 'Douglas Adams',
 'si': 'ඩග්ලස් ඇඩම්ස්',
 'sk': 'Douglas Adams',
 'sl': 'Douglas Adams',
 'sq': 'Douglas Adams',
 'sr': 'Даглас Адамс',
 'sr-ec': 'Даглас Адамс',
 'sr-el': 'Daglas Adams',
 'sv': 'Douglas Adams',
 'sw': 'Douglas Adams',
 'ta': 'டக்ளஸ் ஆடம்ஸ்',
 'tcy': 'ಡಾಗ್ಲಸ್ ಆಡಮ್ಸ್',
 'te': 'డగ్లస్ ఆడమ్స్',
 'th': 'ดั๊กลาส อดัมส์',
 'tr': 'Douglas Adams',
 'uk': 'Дуглас Адамс',
 'ur': 'ڈگلس ایڈم',
 'vec': 'Douglas Adams',
 'vep': 'Adams Duglas',
 'vi': 'Douglas Adams',
 'vls': 'Douglas Adams',
 'vo': 'Douglas Adams',
 'wa': 'Douglas Adams',
 'war': 'Douglas Adams',
 'wo': 'Douglas Adams',
 'zh': '道格拉斯·亚当斯',
 'zh-cn': '道格拉斯·亚当斯',
 'zh-hans': '道格拉斯·亚当斯',
 'zh-hant': '道格拉斯·亞當斯',
 'zh-hk': '道格拉斯·亞當斯',
 'zh-mo': '道格拉斯·亞當斯',
 'zh-my': '道格拉斯·亚当斯',
 'zh-sg': '道格拉斯·亚当斯',
 'zh-tw': '道格拉斯·亞當斯',
 'zu': 'Douglas Adams'}

For convenience, after the itempage.get() is called, the data is stored in the page variable also:

itemdata['labels'] == itempage.labels
True

Exercise - Check whether a given Item has a label in your native language

Find the language code for your native language and write a function to check if a given item has a label in that language, and what other aliases it has in that language.

Claims

Claims Are other wikidtaa pages that are linked to the given item using properties. The 'claims' are stored as another dictionary with the keys as property IDs (P1003, P1005, P1006, etc.) and the value is a list of objects pywikibot.page.Claim.

Hence, the claim is the value of all the properties that have been set for the given item.

itemdata['claims']
{'P1003': [<pywikibot.page.Claim at 0x7f383c753828>],
 'P1005': [<pywikibot.page.Claim at 0x7f383c7b8780>],
 'P1006': [<pywikibot.page.Claim at 0x7f383c7b24e0>,
  <pywikibot.page.Claim at 0x7f383c7b2710>],
 'P1015': [<pywikibot.page.Claim at 0x7f383c73cc18>],
 'P103': [<pywikibot.page.Claim at 0x7f383c74d908>,
  <pywikibot.page.Claim at 0x7f383c74dd68>],
 'P106': [<pywikibot.page.Claim at 0x7f383c702748>,
  <pywikibot.page.Claim at 0x7f383c702898>,
  <pywikibot.page.Claim at 0x7f383c7029e8>,
  <pywikibot.page.Claim at 0x7f383c702b38>,
  <pywikibot.page.Claim at 0x7f383c702f98>,
  <pywikibot.page.Claim at 0x7f383c7050f0>,
  <pywikibot.page.Claim at 0x7f383c705198>,
  <pywikibot.page.Claim at 0x7f383c705470>],
 'P108': [<pywikibot.page.Claim at 0x7f383c8824e0>],
 'P119': [<pywikibot.page.Claim at 0x7f383c757cf8>],
 'P1196': [<pywikibot.page.Claim at 0x7f383c757a90>],
 'P1207': [<pywikibot.page.Claim at 0x7f383c778438>],
 'P1233': [<pywikibot.page.Claim at 0x7f383c757cc0>],
 'P1258': [<pywikibot.page.Claim at 0x7f383c7538d0>],
 'P1263': [<pywikibot.page.Claim at 0x7f383c74d860>],
 'P1266': [<pywikibot.page.Claim at 0x7f383c7b8908>],
 'P1273': [<pywikibot.page.Claim at 0x7f383c763710>],
 'P1284': [<pywikibot.page.Claim at 0x7f383c702470>],
 'P1315': [<pywikibot.page.Claim at 0x7f383c753400>],
 'P136': [<pywikibot.page.Claim at 0x7f383c7539e8>],
 'P1368': [<pywikibot.page.Claim at 0x7f383c7426d8>],
 'P1375': [<pywikibot.page.Claim at 0x7f383c7b8898>],
 'P140': [<pywikibot.page.Claim at 0x7f383c87a2b0>],
 'P1411': [<pywikibot.page.Claim at 0x7f383c75d940>,
  <pywikibot.page.Claim at 0x7f383c75da58>],
 'P1412': [<pywikibot.page.Claim at 0x7f383c747ac8>],
 'P1415': [<pywikibot.page.Claim at 0x7f383c7b8cf8>],
 'P1417': [<pywikibot.page.Claim at 0x7f383c7022b0>],
 'P1442': [<pywikibot.page.Claim at 0x7f383c763908>],
 'P1477': [<pywikibot.page.Claim at 0x7f383c742240>],
 'P1559': [<pywikibot.page.Claim at 0x7f383c7025f8>],
 'P1670': [<pywikibot.page.Claim at 0x7f383c7b2be0>],
 'P1695': [<pywikibot.page.Claim at 0x7f383c7421d0>],
 'P172': [<pywikibot.page.Claim at 0x7f383c7b8630>],
 'P18': [<pywikibot.page.Claim at 0x7f383c872cf8>],
 'P1816': [<pywikibot.page.Claim at 0x7f383c7b8d68>],
 'P19': [<pywikibot.page.Claim at 0x7f383c753940>],
 'P1953': [<pywikibot.page.Claim at 0x7f383c7b8c88>],
 'P20': [<pywikibot.page.Claim at 0x7f383c7784a8>],
 'P2019': [<pywikibot.page.Claim at 0x7f383c74d208>],
 'P21': [<pywikibot.page.Claim at 0x7f383c75dc88>],
 'P213': [<pywikibot.page.Claim at 0x7f383c7abe80>],
 'P214': [<pywikibot.page.Claim at 0x7f383c7057f0>],
 'P2163': [<pywikibot.page.Claim at 0x7f383c7055f8>],
 'P2168': [<pywikibot.page.Claim at 0x7f383c7b8ac8>],
 'P2188': [<pywikibot.page.Claim at 0x7f383c753160>],
 'P2191': [<pywikibot.page.Claim at 0x7f383c73cc88>],
 'P22': [<pywikibot.page.Claim at 0x7f383c73c0b8>],
 'P227': [<pywikibot.page.Claim at 0x7f383c763ac8>],
 'P2387': [<pywikibot.page.Claim at 0x7f383c76d9e8>],
 'P2435': [<pywikibot.page.Claim at 0x7f383c7058d0>],
 'P244': [<pywikibot.page.Claim at 0x7f383c7abcc0>],
 'P25': [<pywikibot.page.Claim at 0x7f383c76d588>],
 'P26': [<pywikibot.page.Claim at 0x7f383c773c88>],
 'P2604': [<pywikibot.page.Claim at 0x7f383c73c048>],
 'P2605': [<pywikibot.page.Claim at 0x7f383c778978>],
 'P2611': [<pywikibot.page.Claim at 0x7f383c75d048>],
 'P2626': [<pywikibot.page.Claim at 0x7f383c742668>],
 'P268': [<pywikibot.page.Claim at 0x7f383c7b2048>],
 'P269': [<pywikibot.page.Claim at 0x7f383c75df98>],
 'P27': [<pywikibot.page.Claim at 0x7f383c742f98>],
 'P271': [<pywikibot.page.Claim at 0x7f383c705710>],
 'P2963': [<pywikibot.page.Claim at 0x7f383c75dfd0>],
 'P31': [<pywikibot.page.Claim at 0x7f383c747e80>],
 'P345': [<pywikibot.page.Claim at 0x7f383c757e48>],
 'P349': [<pywikibot.page.Claim at 0x7f383c702588>],
 'P373': [<pywikibot.page.Claim at 0x7f383c75d748>],
 'P396': [<pywikibot.page.Claim at 0x7f383c7787b8>],
 'P40': [<pywikibot.page.Claim at 0x7f383c74d358>],
 'P409': [<pywikibot.page.Claim at 0x7f383c742128>],
 'P434': [<pywikibot.page.Claim at 0x7f383c705860>],
 'P509': [<pywikibot.page.Claim at 0x7f383c763c50>],
 'P535': [<pywikibot.page.Claim at 0x7f383c76d978>],
 'P551': [<pywikibot.page.Claim at 0x7f383c6fd278>,
  <pywikibot.page.Claim at 0x7f383c6fd470>,
  <pywikibot.page.Claim at 0x7f383c6fd630>],
 'P569': [<pywikibot.page.Claim at 0x7f383c742dd8>],
 'P570': [<pywikibot.page.Claim at 0x7f383c7b8f98>],
 'P646': [<pywikibot.page.Claim at 0x7f383c7b2c50>],
 'P648': [<pywikibot.page.Claim at 0x7f383c753390>],
 'P69': [<pywikibot.page.Claim at 0x7f383c76de48>,
  <pywikibot.page.Claim at 0x7f383cfd3438>],
 'P691': [<pywikibot.page.Claim at 0x7f383c773e48>],
 'P734': [<pywikibot.page.Claim at 0x7f383c7b2240>],
 'P735': [<pywikibot.page.Claim at 0x7f383c763ef0>,
  <pywikibot.page.Claim at 0x7f383c76d1d0>],
 'P800': [<pywikibot.page.Claim at 0x7f383c7b2cc0>,
  <pywikibot.page.Claim at 0x7f383c7b8048>,
  <pywikibot.page.Claim at 0x7f383c7b82e8>,
  <pywikibot.page.Claim at 0x7f383c7b8470>,
  <pywikibot.page.Claim at 0x7f383c7b84e0>],
 'P856': [<pywikibot.page.Claim at 0x7f383c763b38>],
 'P866': [<pywikibot.page.Claim at 0x7f383c7059e8>],
 'P9': [<pywikibot.page.Claim at 0x7f383c753278>],
 'P906': [<pywikibot.page.Claim at 0x7f383c763a58>],
 'P910': [<pywikibot.page.Claim at 0x7f383c6fd7b8>],
 'P947': [<pywikibot.page.Claim at 0x7f383c757978>],
 'P949': [<pywikibot.page.Claim at 0x7f383c742d30>],
 'P950': [<pywikibot.page.Claim at 0x7f383c7abef0>],
 'P998': [<pywikibot.page.Claim at 0x7f383c6fd9e8>,
  <pywikibot.page.Claim at 0x7f383c6fda20>,
  <pywikibot.page.Claim at 0x7f383c6fdb70>,
  <pywikibot.page.Claim at 0x7f383c6fdd30>,
  <pywikibot.page.Claim at 0x7f383c6fdef0>,
  <pywikibot.page.Claim at 0x7f383c7020f0>]}
# Similarly, this is available in the page object using:
itemdata['claims'] == itempage.claims
True

Let's take a look at the P800 (notable work) for the item Q42 (Douglas Adams):

itempage.claims['P1003']
[<pywikibot.page.Claim at 0x7f383c753828>]

There are multiple claims for the property "P800" and we can ask pywikibot to resolve the claim and fetch the data about the claim:

itempage.claims['P800'][0].getTarget()
ItemPage('Q25169')

So, we notice that the claim for the first "notable work (P800)" of "Douglas Adams (Q42)" is the item Q25169. As this is another ItemPage we can fetch the english label for this item by doing:

p800_claim_target = itempage.claims['P800'][0].getTarget()
p800_claim_target.get()
p800_claim_target.labels['en']
"The Hitchhiker's Guide to the Galaxy pentalogy"

So, finally we were able to find one of the most notable work of Douglas Adams using the wikidata API exposed by pywikibot. Imagine doing the same in the english wikipedia !

Thought exercise: How would you figure out the most notable work of the author using the chunk of text given by an English wikipedia Page ?

Exercise - Check whether item is in India

Given a item ID, check whether the Item is in India by checking the value of the "country" property of the item. Write a function that checks this.

Hence, when the function is run on Q987 (New Delhi), it should give True but on Q62 (San Francisco) it should give False.

6. Property Pages

Sometimes, it is important to be able to fetch data about the property we find itself. For example, if we want to list the english label of the property and the value in a tabular form.

To do this, we use a PropertyPage object to deal with properties:

propertypage = pywikibot.PropertyPage(wikidata, 'P512')
propertypage
PropertyPage('Property:P512')

In the PropertyPage, we again can access the data similar to how it was accessed in the ItemPage:

propertypage.get()
propertypage.labels['en']
'academic degree'

4. Wikidata data types

On Wikidata, we've already seen ItemPages and PropertyPages. But sometimes, a Claim's value need not be another Itempage, and can be some other data type like text, number, datetime, etc. Pywikibot provides a class for each of these data-types for easier accesibility to the value and resolve the claim.

The Data types available in wikidata can be seen at: https://www.wikidata.org/wiki/Special:ListDatatypes

The wikidata datatypes provided and the corresponding name of the wikidata data-type are:

  • Item - pywikibot.page.ItemPage - Link to other items at the project.
  • Property - pywikibot.page.PropertyPage - Link to properties at the project.
  • Global Coordinate - pywikibot.Coordinate - Literal data for a geographical position given as a latitude-longitude pair in gms or decimal degrees.
  • Time - pywikibot.WbTime - Literal data field for a point in time.
  • Quantity - pywikibot.WbQuantity - Literal data field for a quantity that relates to some kind of well-defined unit.
  • Monolingual Text - pywikibot.MonoLingualText - Literal data field for a string that is not translated into other languages.

Data types mapping to str

Some types of wikidata have are made specially to show them using a different method for example, showing it as a link, etc. But they all map to the python str. They are:

  • String - str - Literal data field for a string of glyphs. Generally do not depend on language of reader.
  • URL - str - Literal data field for a URL.
  • External Identifier - str - Literal data field for an external identifier. External identifiers may automatically be linked to an authoritative resource for display.
  • Mathematical formula - str - Literal data field for mathematical expressions, formula, equations and such, expressed in a variant of LaTeX.
# Item
item = pywikibot.ItemPage(wikidata, "Q42").get()['claims']['P31'][0].getTarget()
print("Type:", type(item))
print("Instance of Douglas Adams:", item, '(', item.get()['labels']['en'], ')')
Type: <class 'pywikibot.page.ItemPage'>
Instance of Douglas Adams: [[wikidata:Q5]] ( human )
# Property
_property = pywikibot.PropertyPage(wikidata, "Property:P31")
_property.get()
print("Type:", type(_property))
print("Property 'instance of':", _property, '(', _property.labels['en'], ')')
Type: <class 'pywikibot.page.PropertyPage'>
Property 'instance of': [[wikidata:Property:P31]] ( instance of )
# Global Coordinate
coord = pywikibot.ItemPage(wikidata, "Q668").get()['claims']['P625'][0].getTarget()
print("Type:", type(coord))
print("Coordinate location of India:", coord)
Type: <class 'pywikibot.Coordinate'>
Coordinate location of India: {
    "altitude": null,
    "globe": "http://www.wikidata.org/entity/Q2",
    "latitude": 21,
    "longitude": 77,
    "precision": 0.00027777777777778
}
# Time
_time = pywikibot.ItemPage(wikidata, "Q28865").get()['claims']['P571'][0].getTarget()
print("Type:", type(_time))
print("Inception of Python (programming language):", _time)
Type: <class 'pywikibot.WbTime'>
Inception of Python (programming language): {
    "after": 0,
    "before": 0,
    "calendarmodel": "http://www.wikidata.org/entity/Q1985727",
    "precision": 11,
    "time": "+00000001991-02-20T00:00:00Z",
    "timezone": 0
}
# Quantity
qty = pywikibot.ItemPage(wikidata, "Q668").get()['claims']['P1082'][0].getTarget()
print("Type:", type(qty))
print("Population in India:", qty)
Type: <class 'pywikibot.WbQuantity'>
Population in India: {
    "amount": "+1263200000",
    "lowerBound": "+1263200000",
    "unit": "1",
    "upperBound": "+1263200000"
}
# Monolingual text
monolingual_text = pywikibot.ItemPage(wikidata, "Q42").get()['claims']['P1477'][0].getTarget()
print("Type:", type(monolingual_text))
print("Birth name of Douglas Adams:", monolingual_text)
Type: <class 'pywikibot.WbMonolingualText'>
Birth name of Douglas Adams: {
    "language": "en",
    "text": "Douglas No\u00ebl Adams"
}
# String
_string = pywikibot.ItemPage(wikidata, "Q28865").get()['claims']['P348'][0].getTarget()
print("Type:", type(_string))
print("Version of Python:", _string)
Type: <class 'str'>
Version of Python: 2.7.6
# Mathematical Formula
formula = pywikibot.ItemPage(wikidata, "Q11518").get()['claims']['P2534'][0].getTarget()
print("Type:", type(formula))
print("Formula of Pythagorean theorem:", formula)
Type: <class 'str'>
Formula of Pythagorean theorem: a^2+b^2=c^2

Exercise - Find URL and External identifier

Using the API, find the type and value of the Official website (P856) and the Freebase identifier (P646) of Python (Q28865).

5. Adding more meaning to Properties

Frequently, a property value may require additional data. For example, consider the property educated at (P69) in the earlier data fetched from Douglas Adams (Q42). It can be seen on WikiData that "St John's College" is mentioned as one of his education schools from "start time" of 1971 to "end time" of 1974. And it also says his "academic major" is English literature and "academic degree" was Bachelor of Arts.

Qualifiers

All this information would not be found if Wikidata was restricted to a (property, value) storage structure. Hence, Wikidata also allows Qualifiers. Qualifiers expand on the dta provided by a (property, value) pair by giving it context. Qualifiers also consist of a (property, value) !

In the above example, the properties which are being used as qualifiers are:

itempage.claims['P69']
[<pywikibot.page.Claim at 0x7f383c76de48>,
 <pywikibot.page.Claim at 0x7f383cfd3438>]
itempage.claims['P69'][0].getTarget().get()
itempage.claims['P69'][0].getTarget().labels['en']
"St John's College"
itempage.claims['P69'][0].qualifiers
OrderedDict([('P582', [<pywikibot.page.Claim at 0x7f383c773438>]),
             ('P812', [<pywikibot.page.Claim at 0x7f383c773940>]),
             ('P512', [<pywikibot.page.Claim at 0x7f383c773a20>]),
             ('P580', [<pywikibot.page.Claim at 0x7f38445357f0>])])

The qualifiers are again claims, as they are similar to the (property, value) pair for item pages. Let us see what the value of the qualifier is by resolving the claim:

# Fetch the label of the P512 (academic degree) property
claim = itempage.claims['P69'][0]
claim.qualifiers['P512'][0].getTarget().get()
claim.qualifiers['P512'][0].getTarget().labels['en']
'Bachelor of Arts'

Some qualifiers may have a value which is not another item, for example the "start date" qualifier. In such cases, we need to check the type of the item:

claim = itempage.claims['P69'][0]
claim.qualifiers['P580'][0].getTarget()
WbTime(year=1971, month=0, day=0, hour=0, minute=0, second=0, precision=9, before=0, after=0, timezone=0, calendarmodel=http://www.wikidata.org/entity/Q1985727)

Here, WBTime is a pywikibot class which handles the format of WikiBase Time. Wiki base is the underlying technology that powers the structured editing and so on of Wikidata.

Other functions to modify qualifiers are:

  • claim.removeQualifier()
  • claim.addQualifier()
  • claim.has_qualifier()

Exercise - Find time studied at school

In the case of Douglas Adams as we saw, there are 2 schools mentioned. Using the start time and end time, find the number of years that he studdied in any school and print it out in the format:

<school name>: <start year> to <end year> => <n> years

Reference (Sources)

Other than the qualifier, we would also want the source of the data. Hence, the reference or source field helps in adding, removing, editing these:

itempage.claims['P69'][0].getSources()
[OrderedDict([('P248', [<pywikibot.page.Claim at 0x7f383c773470>])]),
 OrderedDict([('P854', [<pywikibot.page.Claim at 0x7f383c773588>]),
              ('P364', [<pywikibot.page.Claim at 0x7f383c7735c0>]),
              ('P123', [<pywikibot.page.Claim at 0x7f383c773668>]),
              ('P813', [<pywikibot.page.Claim at 0x7f383c773828>]),
              ('P1476', [<pywikibot.page.Claim at 0x7f383c773860>])])]

Again, the source is a (property, value) where the property describes what type of source it is. Some popular properties are:

A source is again a list of (property, value) tuples. It can have additional properties like: "original language of work", "publisher", "author", "title", "retrieved", etc. if necessary.

Let us take a look at a source here:

source = itempage.claims['P69'][0].getSources()[0]
source
OrderedDict([('P248', [<pywikibot.page.Claim at 0x7f383c773470>])])
# Get the value of the first tuple in the source:
source['P248']
[<pywikibot.page.Claim at 0x7f383c773470>]
source['P248'][0].getTarget().get()['labels']['en']
'Encyclopædia Britannica Online'

Exercise - Check number of values that have a source as English Wikipedia

A large number of data in wikidata was pulled from the "English Wikipedia". Go through all (property, value) pairs and check how many of them were taken from the English Wikipedia (Q328).

7. Search wikidata using python

The Search on wikidata can be triggered using the python api too. To do this, we use the pywikibot.data.api.Request class which provides with helper functions to query the wikidata website and fetch results:

india_search = pywikibot.data.api.Request(
    site=wikidata,
    parameters={"action": "wbsearchentities",
                "format": "json",
                "type": "item",
                "language": "en",
                "search": "India"})

india_search.submit()
{'search': [{'concepturi': 'http://www.wikidata.org/entity/Q668',
   'description': 'federal republic in South Asia',
   'id': 'Q668',
   'label': 'India',
   'match': {'language': 'en', 'text': 'India', 'type': 'label'},
   'pageid': 966,
   'title': 'Q668',
   'url': '//www.wikidata.org/wiki/Q668'},
  {'concepturi': 'http://www.wikidata.org/entity/Q1488929',
   'description': 'Wikipedia disambiguation page',
   'id': 'Q1488929',
   'label': 'India',
   'match': {'language': 'en', 'text': 'India', 'type': 'label'},
   'pageid': 1426615,
   'title': 'Q1488929',
   'url': '//www.wikidata.org/wiki/Q1488929'},
  {'concepturi': 'http://www.wikidata.org/entity/Q1936198',
   'description': '2005 album by German band Xandria',
   'id': 'Q1936198',
   'label': 'India',
   'match': {'language': 'en', 'text': 'India', 'type': 'label'},
   'pageid': 1865814,
   'title': 'Q1936198',
   'url': '//www.wikidata.org/wiki/Q1936198'},
  {'concepturi': 'http://www.wikidata.org/entity/Q2060630',
   'description': 'cat owned by USA presidential Bush family',
   'id': 'Q2060630',
   'label': 'India',
   'match': {'language': 'en', 'text': 'India', 'type': 'label'},
   'pageid': 1987980,
   'title': 'Q2060630',
   'url': '//www.wikidata.org/wiki/Q2060630'},
  {'concepturi': 'http://www.wikidata.org/entity/Q6019245',
   'description': 'album by Vega',
   'id': 'Q6019245',
   'label': 'India',
   'match': {'language': 'en', 'text': 'India', 'type': 'label'},
   'pageid': 5800903,
   'title': 'Q6019245',
   'url': '//www.wikidata.org/wiki/Q6019245'},
  {'concepturi': 'http://www.wikidata.org/entity/Q16429066',
   'description': 'female given name',
   'id': 'Q16429066',
   'label': 'India',
   'match': {'language': 'en', 'text': 'India', 'type': 'label'},
   'pageid': 18056127,
   'title': 'Q16429066',
   'url': '//www.wikidata.org/wiki/Q16429066'},
  {'concepturi': 'http://www.wikidata.org/entity/Q17055962',
   'description': 'passage by Herodotus',
   'id': 'Q17055962',
   'label': 'India',
   'match': {'language': 'en', 'text': 'India', 'type': 'label'},
   'pageid': 18661696,
   'title': 'Q17055962',
   'url': '//www.wikidata.org/wiki/Q17055962'}],
 'search-continue': 7,
 'searchinfo': {'search': 'India'},
 'success': 1}

As you can see, this function simply returns the dictionary search results directly. This needs to be parsed to be more useful. By modifying the parameters we can also search for the item using other languages and types.

Exercise: Create an ItemPage for every search result

The search result given by the API is a python dictionary. But it has a lot of data which may not be very useful to us.

  1. Loop over every search item and create a ItemPage object and store these in a list.
  2. Finally, loop over the ItemPage list and print the english label for each item in the search using the ItemPage class.

8. Further Reading

To read more ways of using pywikibot to access wikidata, go to https://www.wikidata.org/wiki/Wikidata:Pywikibot_-_Python_3_Tutorial

SparQL queries are SQL like queries that can be run on Wikidata to fetch data from it. To try out SparkQL queries and visualize the data using nice plots, you can use https://query.wikidata.org. It has a lot of example SparQL queries which can be useful to learn SparQL.