Banana
9c7cf993e5
some improvements to the imdb dataset to mysql importer
|
3 years ago | |
|---|---|---|
| .. | ||
| datasets | 4 years ago | |
| lib | 3 years ago | |
| README | 3 years ago | |
| TODO | 3 years ago | |
| api.php | 3 years ago | |
| import.php | 3 years ago | |
README
https://www.imdb.com/interfaces/
Subsets of IMDb data are available for access to customers for personal and non-commercial use.
You can hold local copies of this data, and it is subject to our terms and conditions.
Please refer to the Non-Commercial Licensing
https://help.imdb.com/article/imdb/general-information/can-i-use-imdb-data-in-my-software/G5JTRESSHJBBHTGX
and copyright/license and verify compliance.
https://www.imdb.com/conditions
This will import the imdb dataset tsv into your mysql database for further user.
Code based on the dataset at feb. 2020
There will be no relations or whatsoever. Just plain data into tables.
It also does not create any relation tables yet. Some tables have columns which have
strings separated by comma in them.
As of march 2020
Title crew looks strange. The longest line is 16313 (wc -L title.crews.tsv)
therefore the column directors and writers are defined as text and not
varchar. Do not know if this is an error or correct...
This is not a good example to be written in PHP. But you can use it.
Don't execute it through a webserver. It is a CLI script
# Usage
Download and place the tsv files from https://www.imdb.com/interfaces/ into the datasets folder.
Decide which one do you need. Alter $filesToImport in import.php to match the files.
Decide if you need a full text search index. Needed if you want to use the api.php.
Adding the index after the initial import is not a good idea. It takes ages!!
Using the index will slow down the import. To use change BUILD_INDEX to true in import.php file
Subsets of IMDb data are available for access to customers for personal and non-commercial use.
You can hold local copies of this data, and it is subject to our terms and conditions.
Please refer to the Non-Commercial Licensing
https://help.imdb.com/article/imdb/general-information/can-i-use-imdb-data-in-my-software/G5JTRESSHJBBHTGX
and copyright/license and verify compliance.
https://www.imdb.com/conditions
This will import the imdb dataset tsv into your mysql database for further user.
Code based on the dataset at feb. 2020
There will be no relations or whatsoever. Just plain data into tables.
It also does not create any relation tables yet. Some tables have columns which have
strings separated by comma in them.
As of march 2020
Title crew looks strange. The longest line is 16313 (wc -L title.crews.tsv)
therefore the column directors and writers are defined as text and not
varchar. Do not know if this is an error or correct...
This is not a good example to be written in PHP. But you can use it.
Don't execute it through a webserver. It is a CLI script
# Usage
Download and place the tsv files from https://www.imdb.com/interfaces/ into the datasets folder.
Decide which one do you need. Alter $filesToImport in import.php to match the files.
Decide if you need a full text search index. Needed if you want to use the api.php.
Adding the index after the initial import is not a good idea. It takes ages!!
Using the index will slow down the import. To use change BUILD_INDEX to true in import.php file