Connection issue

chris's Avatar

chris

27 Dec, 2017 08:30 PM

Starting on December 21st our systems search hasn't been working correctly with errors similar to: "Lost connection to MySQL server at 'reading authorization packet', system error: 0 - SELECT * FROM `user_tenant53_core` WHERE MATCH('client') AND `role_ids` IN (4) AND `sphinx_deleted` = 0 LIMIT 0, 8; SHOW META". it appears that our sphinx instance isn't running according to your dashboard. Would you be able to bring our search back up?

  1. Support Staff 1 Posted by Pat Allan on 28 Dec, 2017 12:10 AM

    Pat Allan's Avatar

    Hi Chris,

    Sorry about this! I’ve just got the daemon up and running, though I’m not quite sure why it wasn’t. I do have monit in place tracking all Sphinx daemons, so normally if there are problems I’m much quicker at fixing them. For some reason your daemon didn’t have the monit configuration, hence it wasn’t brought back up immediately after the downtime.

    The config is only removed when the stop command is run… but I’d be surprised if that happened and there was no corresponding start afterwards. I’ll need to go through the command history for your app, which can take a while.

    In the meantime: if you notice this happen again, you can just run `heroku run bundle exec flying-sphinx start` to get the daemon running again, instead of waiting for me to be online and awake.

    Regards,


    Pat

  2. Support Staff 2 Posted by Pat Allan on 28 Dec, 2017 12:41 AM

    Pat Allan's Avatar

    Just further to this: it seems a rebuild command was issued on the 21st, and only the final component of that - the starting of the daemon - failed, though I’ve got no data as to why. I’ve made a change to get far better debugging information on such situations, so if this crops up again I can look into why more deeply, and also just be better at getting daemons running again more quickly.

    > On 28 Dec 2017, at 11:10 am, Pat Allan <[email blocked] <mailto:[email blocked]>> wrote:
    >
    > Hi Chris,
    >
    > Sorry about this! I’ve just got the daemon up and running, though I’m not quite sure why it wasn’t. I do have monit in place tracking all Sphinx daemons, so normally if there are problems I’m much quicker at fixing them. For some reason your daemon didn’t have the monit configuration, hence it wasn’t brought back up immediately after the downtime.
    >
    > The config is only removed when the stop command is run… but I’d be surprised if that happened and there was no corresponding start afterwards. I’ll need to go through the command history for your app, which can take a while.
    >
    > In the meantime: if you notice this happen again, you can just run `heroku run bundle exec flying-sphinx start` to get the daemon running again, instead of waiting for me to be online and awake.
    >
    > Regards,
    >
    > —
    > Pat
    >
    >> On 28 Dec 2017, at 7:32 am, chris <[email blocked] <mailto:[email blocked]>> wrote:
    >>

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac