Can One MySQL DB hold web site contents and index each web site?

Mike Dawn's Avatar

Mike Dawn

28 Jan, 2017 11:35 PM

Hello,
I am a new user of Sphinx. I am trying to scrap many web sites and storage into single DB. I would like to run indexer DB contents into each single web site index file. User can search web content from specific web site index during web engine search. Can this be done in Sphinx?

Thank You
Mike Dawn

  1. Support Staff 1 Posted by Pat Allan on 29 Jan, 2017 01:03 AM

    Pat Allan's Avatar

    Hi Mike,

    Once the data is in a MySQL or PostgreSQL database - however it’s sourced - it can be indexed by Sphinx.

    I’ve actually written a service similar to what you’re thinking - it takes in HTML data from static Jekyll-generated sites, and makes them searchable with Sphinx. The code is open sourced as well - if you’re familiar with Ruby it may be useful inspiration:
    https://github.com/pat/drumknott-server <https://github.com/pat/drumknott-server>

    It’s worth noting that in this case I’m using real-time indices, which means I’m communicating the data updates via Sphinx’s SphinxQL protocol, rather than using Sphinx’s indexer CLI tool. Either approach would work fine in your case.

    Regards,


    Pat

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac