Privacy of indexed data!

Hasssan's Avatar


08 Dec, 2015 12:41 PM


I was wondering about the privacy of the data shared on flying sphinx's servers. Is there a privacy policy page and/or terms and conditions about Customer Data?


  1. Support Staff 1 Posted by Pat Allan on 09 Dec, 2015 02:42 AM

    Pat Allan's Avatar

    Hi Hassan

    There isn’t a privacy or terms page - but you’re right, there really should be, so I’ll work on adding that.

    From a general security perspective:

    * Each Flying Sphinx customer has an identifier and an API key generated when they register (as you’ll see in your heroku config environment variables). These are unique per Heroku app, and the identifier is how I separate each customer’s data.

    * All external API calls (e.g. via the flying-sphinx gem) are authenticated using both the identifier and the API key. All internal API calls are done with different API keys (and are different endpoints as well, to keep things compartmentalised).

    * When customers upload configuration files, the API modifies those configurations to set all paths to keep all index files and other files (binlogs, wordforms, etc) separated on a per-app basis. So, it doesn’t matter what paths the customer sends through in their configuration, Flying Sphinx will ensure they’re appropriate.

    * Sphinx itself doesn’t have any form of authentication for connections - which is a little frustrating - but Flying Sphinx runs a Sphinx proxy on each server to provide a basic level of authentication. This is done by a patch I wrote on the older binary protocol, or via the MySQL username when connecting via Sphinx's MySQL protocol. The authentication uses both the identifier and API key, so even if/when customers share indexing logs (generally for debugging/support purposes) which might display paths (thus, the identifier), connection details are still secret because the API key is not revealed.

    * When customers remove the add-on, all records for that customer (configuration files, index files, indexing logs, etc) are deleted.

    Hope these details are useful - if you think there’s anything I’ve missed, do let me know!

    Kind regards,

    Pat Allan

  2. 2 Posted by Hassan Rehman on 09 Dec, 2015 09:32 AM

    Hassan Rehman's Avatar


    Thank you Pat. From what I understand, the channels for connecting the app
    to the server are encrypted and protected. I'm concerned about the privacy
    of the data that's stored. However difficult it is to redo data from
    whatever form it is stored in, is it encrypted? and apart from the app and
    yourself, who has access to API keys?


  3. Support Staff 3 Posted by Pat Allan on 09 Dec, 2015 12:19 PM

    Pat Allan's Avatar

    Hi Hassan

    The index files are not encrypted by myself/Flying Sphinx, but I’m not aware of anything that can reverse-engineer Sphinx index files into the original data (granted, that certainly doesn’t mean it doesn’t exist).

    However, the index files are not available publicly to anyone - the Flying Sphinx servers are pretty tightly locked down, and SSH access is only available via known keys. The files are backed up to S3 regularly, but that is of course kept private as well.

    For app credentials (the identifier and API key), Heroku is the only organisation who has access to them beyond myself/Flying Sphinx, and you. For all internal server communication within Flying Sphinx, those API keys aren’t shared with any third parties at all.



  4. 4 Posted by Hassan Rehman on 09 Dec, 2015 12:29 PM

    Hassan Rehman's Avatar

    Thank you Pat that's all the information i needed. The rest lies with the
    security team to decide.

    Thanks for all your help.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:


Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts


? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac