Starting two hours ago the Sphinx deamon keeps crashing for our app "doghouseapi" with authorization errors into the MySQL database on cleardb, but every other part of the App is able to connect as we well as test rake, and Sequel Pro.
2016-02-29T20:38:48.885400+00:00 app[web.1]: 2016-02-29 20:38:48 +0000: Rack app error: #<ThinkingSphinx::SphinxError: Lost connection to MySQL server at 'reading authorization packet', system error: 0 - SELECT * FROM `book_core`, `book_delta` WHERE MATCH('@isbn 9783865399724') AND `lang` = 2106599819 AND `comic` = 0 AND `sphinx_deleted` = 0 ORDER BY `lang` DESC, weight() DESC, `id` ASC LIMIT 0, 25 OPTION ranker=sph04; SHOW META>
Tried to migrate to a bigger plan to work around the issue, but get errors from Heroku saying it can't connect to the Vendor to upgrade the plan.
No sure if it was a patch on your side recently or the Heroku patch that started the issue. We haven't updated in weeks, but rolled back for testing and that did not fix the issue.
Also can't run a index rake.
spec/spec.rake: cannot load such file -- rspec/core/rake_task
I, [2016-02-29T20:25:40.491442 #3] INFO -- : Executing Action: restart
W, [2016-02-29T20:26:40.492113 #3] WARN -- : Action timed out. If this is happening regularly, please contact Flying Sphinx support: http://support.flying-sphinx.com
I, [2016-02-29T20:26:40.492495 #3] INFO -- : Action Finished: restart
Thanks for looking into the issue, let me know if you need any more data.
#1 Posted by jack on 29 Feb, 2016 09:24 PM
Thanks. Everything is back up, including deltas and Index. Not sure if it was a Heroku configuration issue, but thanks for taking the time to look.
jack closed this discussion on 29 Feb, 2016 09:25 PM.
Pat Allan re-opened this discussion on 29 Feb, 2016 10:01 PM
#2 Posted by Pat Allan Support staff on 29 Feb, 2016 10:01 PM
Jack, definitely my fault, not a Heroku configuration issue: http://status.flying-sphinx.com/incidents/6g18cc3cgcnl
Appreciate your patience, and very sorry for the outage. If you see any issues persisting, do let me know.
#3 Posted by jack on 29 Feb, 2016 10:39 PM
Thank you Pat, you are always on top of things. I am just glad you wake up early. As you also seem to be up late at night. :)