|Minimum required ownCloud 8.1|
Maximum required ownCloud 9.0
|Submitted: Dec 10 2014|
Updated: Apr 4 2016
OPDS is, according to the standard (http://opds-spec.org/specs/opds-catalog-1-1-20110627/), '...a syndication format for electronic publications based on Atom and HTTP. OPDS Catalogs enable the aggregation, distribution, discovery, and acquisition of electronic publications...'. In short, it makes it possible to access publications (which are not limited to electronic books) on any OPDS client device. Examples of the latter are phones and tablets running OPDS-compatible book readers, eg FBReader/CoolReader/Aldiko on Android or Marvin/KyBook/Stanza on iOS. enable/disable the feed (it is disabled by default)
The OPDS catalog app enables Owncloud users to publish a sub-tree of their personal filesystem as an OPDS feed. Since Owncloud currently has limited to no support for metadata, these are for now stored in a separate table. As of v0.3 OPDS catalog can extract all relevant metadata from EPUB documents. v0.5 introduced ISBN-based metadata retrieval, while Calibre-generated metadata.opf files are parsed since v0.6.
If the directory in which the file is found contains a valid metadata.opf file, this file is parsed and all relevant Dublin Core metadata is applied to all files in the directory. If a cover is specified in this file, and the relevant file is found in the directory, this cover is used as preview and thumbnail for all files in the directory. If the metadata is invalid (ie. one or more of title, author and language are not defined) the files themselves are parsed for metadata in the 'normal' way.
Keep in mind that publications for which valid metadata has already been cached in the database will not be scanned again until a rescan is scheduled. This can be done in the personal settings page, use the 'Schedule rescan' button.
If an ISBN is found in either existing metadata or in the first 10 pages of the publication, metadata is retrieved from ISBNdb (key required, http://isbndb.com/account/logincreate, max. 500 queries/day) and Google Books (no key required, but rate limit is higher (1000/day) when one is used).
Documents for which no metadata is found will appear with sparse entries for now: only title (as in 'filename'), file size, cover image (where available), modification time and content links are provided. The first visit to a new (or updated) directory will take a bit longer while OPDS catalog scans for metadata, subsequent visits will be faster due to the use of cached metadata.
The OPDS root feed links to a hierarchical navigation feed mirroring the directory structure as well as a 'personal bookshelf' containing links to all downloaded books (most recent download listed first). This 'personal bookshelf' will be empty (0 books) at first. Use the 'Browse catalog' link to download a book and it'll appear on the 'personal bookshelf'. Download another, and it will appear above the one you previously downloaded. This makes it possible to get at books you are in the process of reading from different devices, or to easily re-visit a book you downloaded earlier.
The feed is in compliance with the OPDS 1.1 specification according to the online OPDS validator (http://opds-validator.appspot.com/).
In the personal settings page there are options to:
set the feed title (default is "Full Name's Library"
set the feed root directory (the default is /Library)
enter a list of extensions to which the feed should be limited (by default this field is empty so it publishes all files descending from the feed root)
enter a list of file names to be skipped (default is 'metadata.opf,cover.jpg')
schedule a rescan of all metadata. This will only set the rescan date to 'now' for all entries, the actual rescan is performed on a by-publication base the next time that publication is visited
clear the personal bookshelf (this only clears the list, it does not delete any books)
The admin settings page contains options to:
set the feed subtitle (default is 'Owncloud OPDS catalog' where 'Owncloud' is the instance name of the OC installation)
change file preview preferences (which should probably be in core or in a separate app as this changes a system-wide setting ('enabledPreviewProviders'))
change the cover image and thumbnail dimensions. By default these are square and rather small (200x200 for covers, 36x36 for thumbnails); If you want them in more book-like proportions change these to something with a width:height ratio of around 1:1.6. 300x480 for covers and 36x58 for thumbnails is a good compromise between image download size (~20KB-40KB jpeg) and quality. What you enter here is purely a personal preference, there is no standard size or ratio. Just make sure that the thumbnails are small-ish as these are often displayed as icons in file lists.
Enter ISBNdb key to use for metadata retrieval, leave blank if you don't have (or want) a key. In that case it only uses Google Books (which can be used with or without a key) for ISBN lookup.
Enter Google Books API key to use for metadata retrieval, leave blank if you don't have (or want) a key. Google Books API works without a key, but the rate limit is higher (1000 lookups per day) when a key is used.
The OPDS feed is disabled when the app is installed, enable it in the personal settings page under the header 'OPDS'. Every user has his/her own feed, which feed you get depends on which credentials you enter.
To connect to the OPDS feed, point your OPDS client at the app URL:
If all goes well, the client should ask for a username and password - enter your Owncloud credentials here. You should use HTTPS to avoid compromising your credentials (something which goes for Owncloud in general).
OPDS catalog is hosted on github: https://github.com/Yetangitu/owncloud-apps
Tested OPDS clients
FBReader on Android: OK
Aldiko on Android: OK (does not show cover images, probably Aldiko bug: https://github.com/rvolz/BicBucStriim/issues/80
CoolReader on Android: OK (CoolReader browser adds extraneous '/' characters to the URL, probably related to this bug: http://sourceforge.net/p/crengine/bugs/267/, this resulted in failed downloads in earlier versions, works as of OC8.1/OPDS catalog 0.6.8)
Moon+ Reader on Android: OK
KyBook on iOS: OK
Marvin on iOS: OK
eBook Search on iOS: browsing works, downloading does not (401 error, 'Unauthorised')
Gecko-based browsers: OK, feed can be browsed and books can be downloaded without additional software
Freda on Windows (Phone & 8.1): OK (mconway, see comments)
About the appcodechecker
OPDS catalog makes use of several 'private' Owncloud methods for which no 'public' alternatives are available. One of the side effects of this is that the 'appcodechecker' will block the automatic installation of this app. As this 'appcodechecker' (in its current incarnation) is no more than a rather naive grep (search) of app code for a list of prefixes, many of which have no 'public' counterpart, this does not make much sense.
My advice is to disable appcodechecker. To be really useful, the app code checker should be implemented in such a way that it warns the developer (and, if possible, points out alternatives for the deprecated calls). There are some other things it could do like check for consistency in the appinfo section (version numbers, name, etc.).
So, for those of you who want their apps to install automagically: add 'appcodechecker' => false, to your configuration file.
0.6.15And yet anther OC9.0+ compatibility fix: changed fileId-handling, explicitly handle command strings in id parameter.
Yet anther OC9.0+ compatibility fix: \OC_Helper -> OCP\Util - this kept the personal settings page from displaying.
Remove unsupported L10N statement which kept files_opds from working in OC9.0
Limit overly-long copyright statements to 256 characters to guard against overflow.
OC_Log is deprecated, changed to OCP\Util::writeLog
Set 'shipped' parameter in info.xml to 'false' to allow direct installation in OC8.2+. Before, this was set to 'true' to keep OC from removing all app data when the app was disabled.
Doubled the maximum allowed length of various database columns, some publications seem to have rather long titles and author lists. A better solution would be to just use the 'text' datatype without length restrictions but this would lead to worse performance in MySQL (PostgreSQL does not have this problem).
Hopefully for the last time: Changed the way the ePub preview provider is registered. This now uses the IProvider interface which is only available from Owncloud 8.1 and onwards, ie. this version does NOT work in earlier versions of Owncloud! If you use an older version of Owncloud and can not (or do not want to) upgrade, download files_opds v0.6.5 (https://github.com/Yetangitu/owncloud-apps/blob/master/dist/files_opds-0.6.5.tar.gz?raw=true) instead
Changed the way the ePub preview provider is registered - again, this is needed to get files_opds working with Owncloud 8.1...
Changed the way the ePub preview provider is registered, this is needed to get files_opds working with Owncloud 8.1
Malformed ISBN13 strings (containing to many dashes, etc) would silently be truncated to 16 characters on MySQL. On Sqlite the malformed value would be stored as-is since it ignores field length statements altogether. PostgreSQL would produce an ERROR here, which is how the problem was identified: https://github.com/Yetangitu/owncloud-apps/issues/7. This is fixed by stripping all but [0-9xX] from the ISBN before storing it. If you use MySQL, you can either schedule a rescan of all data (in the admin settings page) or specifically for truncated records by running the query mentioned in my answer to the bug report (https://github.com/Yetangitu/owncloud-apps/issues/7#issuecomment-74787070). Sqlite users can also run the (sqlite-specific) query from that answer but it is not really necessary as sqlite never truncates text fields (http://www.sqlite.org/faq.html#q9).
A Google Books API key can be entered in the admin settings page. Even though metadata retrieval from Google works without a key, the rate limit is much less strict (1000/day) when a key is used.
Explicitly trim 'description' field in metadata to less than 2048 characters, otherwise Postgresql complains ('value too long for type character varying(2048)')
Get rid of foreach-related log spam
Owncloud complained about OPDS catalog 'not being compatible with sqlite' because it was listed as 'sqlite3' in info.xml. This has been fixed. This is mostly cosmetic, as OPDS catalog did in fact work with sqlite, warning notwithstanding...
Ported to Owncloud 8, should work with OC7 as well - please test
Add option to schedule a rescan of all metadata in the personal settings page. This will only set the rescan date to 'now' for all entries, the actual rescan is performed on a by-publication base when that publication is visited the next time
Calibre-generated metadata (contained in metadata.opf files) and cover images are now used when found in lieu of other metadata sources. If no Calibre-generated metadata is found or the metadata is found to be invalid (defined as omitting either of title, author or language) the files themselves are scanned for metadata.
Potential issue with MySQL-incompatible datetime format in $meta['updated'] fixed.
Fix potential issue (https://github.com/Yetangitu/owncloud-apps/issues/3) with localisation.
Fix potential issue with empty bookshelf
The order in which ISBN metadata providers is queried has been changed, now Google is asked first, then ISBNdb. This way, the daily quotum at ISBNdb lasts longer.
The ISBN metadata parser silently ignored Google data, this is fixed now.
Improved ISBN regexes and regex scanning order to minimise false positives
First ISBN regex always returned negative due to use of unsupported nested character class
Fixed excessive rescans due to faulty time calculation (missing strtotime()). OPDS catalog is a lot faster now...
Improved ISBN regexes, can use further improvement still.
ISBN metadata retrieval from ISBNdb (key required, http://isbndb.com/account/logincreate, max. 500 queries/day) and Google Books (no key required). This either uses a known ISBN (from existing metadata) or scans the first 10 pages of the publication for an ISBN. This does not work when using a Windows server due to the use of external programs (pdftotext, pdfinfo) for pdf to text conversion and metadata collection.
Primary key was not defined for database, leading to the possibility of duplicate records. This is now fixed, if duplicate records are found in the database the upgrade (occ upgrade) will not succeed. Should this happen, the easiest solution is to drop the OPDS table (*PREFIX*opds_metadata) as it will be recreated during the upgrade and -populated when used.
Feed title (personal setting) and subtitle (admin setting) can be configured
Feed now contains atom:content for publications with full metadata, atom:summary for those without. In both cases, some metadata is added (file size, type and name). As the feed also contains dcterms:extent entries (describing file size) this might lead to some duplication of this information in some OPDS clients (eg. KyBook on iOS) but this is preferable over this information not being available in others which do not render dcterms:extent (eg. FBReader)
feed template split into parts to avoid duplication
better layout personal and admin settings pages
openoffice -> opendocument in preview settings
atom:updated in feed header was set to the beginning of the epoch. This could cause problems with new acquisitions not showing up. For now, it is set to the page request time, ie. the feed is 'always' updated. Eventually this should be set to the last modification time of the feed tree
atom:updated in navigation feed was set to 'strange' times due to a miscalculated modification time (ms instead of s), this is now fixed.
Images and thumbnails are now served through OPDS catalog to reduce the size of the feed (shorter URLs)
Full metadata support for EPUB documents. If the document contains relevant metadata, it will appear in the feed.
Metadata is cached in the database using an app-specific table (*dbprefix*opds_metadata). If/when Owncloud gets better metadata support, this table can be dropped
Feed 'entry' nodes are now 'complete' (including metadata)
Added a rudimentary epub parser for metadata extraction
Added epub preview provider (extract cover images from epub documents)
Added cover images (where available, uses normal preview providers and the above-mentioned epub provider)
Added file preview configuration option in admin settings. As this is a system-wide setting, this should really be in core or in a separate app (maybe later).
Added some metadata to feed template (file size, type and filename)
The feed is now in full compliance with the OPDS 1.1 specification, at least according to the online OPDS validator (http://opds-validator.appspot.com/)
Added link URL to personal settings page
This is the initial version. It provides a basic OPDS feed, tested with FBReader and CoolReader on Android. The feed can also be browsed using any Gecko-based browser (Firefox, Seamonkey et al).
The current version of the app is, as stated, rather rudimentary. In a future version the app might be implemented as a 'remote' service. If this happens, the URL to access the feed will change. As it stands now I decided to keep it as simple as possible.