NexVeridian/wikidata-to-surrealdb
1
0
Fork 0
mirror of https://github.com/NexVeridian/wikidata-to-surrealdb.git synced 2025-09-02 01:49:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
A tool for converting Wikidata dumps to a SurrealDB database. Either From a bz2 or json file.
11 commits 1 branch 0 tags 1.4 MiB
Find a file
NexVeridian 8b3311ff21
docker
2023-12-15 00:45:08 -08:00
.cargo init 2023-12-13 01:04:51 +00:00
.devcontainer init 2023-12-13 01:04:51 +00:00
.vscode mulit line json 2023-12-13 15:20:48 -08:00
src docker 2023-12-15 00:45:08 -08:00
.dockerignore docker 2023-12-15 00:45:08 -08:00
.gitignore bz2 2023-12-15 06:47:29 +00:00
Cargo.toml docker 2023-12-15 00:45:08 -08:00
CONTRIBUTING.md readme 2023-12-15 07:55:27 +00:00
docker-compose.dev.yml docker 2023-12-15 00:45:08 -08:00
docker-compose.yml docker 2023-12-15 00:45:08 -08:00
DockerFile readme 2023-12-15 07:55:27 +00:00
LICENSE-Apache init 2023-12-13 01:04:51 +00:00
LICENSE-MIT init 2023-12-13 01:04:51 +00:00
README.md readme 2023-12-15 07:55:27 +00:00
Useful queries.md readme 2023-12-15 07:55:27 +00:00

README.md

A tool for converting Wikidata dumps to a SurrealDB database. Either From a bz2 or json file format.

Getting The Data

https://www.wikidata.org/wiki/Wikidata:Data_access

From bz2 file (Recommended) ~80GB

Dump: Docs

Download - latest-all.json.bz2

From json file

Linked Data Interface: Docs

https://www.wikidata.org/wiki/Special:EntityData/Q60746544.json
https://www.wikidata.org/wiki/Special:EntityData/P527.json

Example .env

DB_USER=root
DB_PASSWORD=root
WIKIDATA_LANG=en
FILE_FORMAT=bz2
FILE_NAME=data/latest-all.json.bz2

How to Query

See Useful queries.md

Table Layout

Thing

pub struct Thing {
    pub table: String,
    pub id: Id,
}

Table: Entity, Property, Lexeme

pub struct EntityMini {
    pub id: Option<Thing>,
    pub label: String,
    pub claims: Thing,
    pub description: String,
}

Table: Claims

pub struct Claims {
    pub id: Option<Thing>,
    pub claims: Vec<Claim>,
}

Table: Claim

pub struct Claim {
    pub id: Thing,
    pub value: ClaimData,
}

ClaimData

pub enum ClaimData {
    Thing(Thing),
    ClaimValueData(ClaimValueData),
}

Similar Projects

License

All code in this repository is dual-licensed under either License-MIT or LICENSE-APACHE at your option. This means you can select the license you prefer. Why dual license.