Heroku Flying Sphinx changes status to Not Running randomly

Popa Daniel's Avatar

Popa Daniel

24 Aug, 2023 07:16 AM

Hi, we have a Rails application hosted on Heroku and using Flying Sphinx add-on. We are using Ruby 2.7.2, Gems versions are: Rails 6.0.6.1, flying-sphinx 2.2.0, and thinking-sphinx 5.5.1. We are using version: 2.2.11 set in config/thinking_sphinx.yml.

On 20.06.2023 the flying_sphinx updated FLYING_SPHINX in Heroku (no release log available); since then, the sphinx status keeps changes to Not Running randomly (~ twice a day). When trying to start the daemon with `bundle exec rake ts:start` I would get `Failed to start searchd daemon.` error message. The daemon starts only after a stop/start cycle, right now we are running a script every 10 mins that starts it when it's down.

Other useful info:
- we are merging the indexes every day via a cronjob

What do you recommend to do in this case?

  1. Support Staff 1 Posted by Pat Allan on 24 Aug, 2023 11:32 PM

    Pat Allan's Avatar

    Hi Daniel,

    I'm very sorry for the interruptions to the service.

    On the 20th, the primary server that your Sphinx daemon on started having availability issues, so I switched it over to the fallback server (which is why the FLYING_SPHINX variables were updated with the new IP address). Unfortunately, that migration hasn't quite gone to plan - a combination of a variety of issues, but I'm gradually working my way through them.

    I've just spent some time on some particular changes that should help your Sphinx daemon become more reliable again. I'll be keeping an eye on things, but do let me know if you spot any further issues. I'm hoping you won't need that script running every 10 minutes.

    Again, my apologies for the issue!

    Kind regards,

    Pat Allan

  2. 2 Posted by Popa Daniel on 30 Aug, 2023 01:18 PM

    Popa Daniel's Avatar

    Hi Pat,

    Thank you for the detailed response. It's been almost a week and everything seems fine. The daemon is up and running, and no random stops happened.

    One thing we noticed a little later, was that the indexer stopped as well, we had to reindex all our data. Everything is fine now!

    Thank you!

  3. Support Staff 3 Posted by Pat Allan on 30 Aug, 2023 01:42 PM

    Pat Allan's Avatar

    Thanks for the update Daniel - things had been seeming fine on my side too, great to know that matches with what you've been seeing as well :)

    Do let me know if any other issues crop up though!

  4. Pat Allan closed this discussion on 30 Aug, 2023 01:42 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac