Flying Sphinx connection being reset
Thomas Fisher
Hello,
We're using Flying Sphinx in production, and are having some with slow performance/dropped connections. In general, the performance seems to be fairly erratic, as you can tell from the attached New Relic graph. Occasionally, search requests will spike, and requests to update the index automatically will raise Errno::ECONNRESET.
We've tried cutting down the number of delta indexes by being selective about what data has changed, and doing a similar strategy for ThinkingSphinx's ActiveRecord AttributeUpdates module, but it doesn't seem to have much of an effect.
If anyone has any experience in dealing with an issue like this, or could point me in the right direction, I'd greatly appreciate it! Thanks!
Flying_Sphinx_API_Response_Times.png
Support Staff 2 Posted by Pat Allan on 26 Apr, 2012 11:58 AM
Hi Thomas
Sorry for not getting back to you sooner. The performance is certainly not ideal, although it's worth noting that flying-sphinx.com is not where your search daemon is - so, only indexing requests and Rails start-up should invoke calls to that server.
That said, search performance may not be brilliant either, which may be related to the fact that the Sphinx index files have been stored on an EBS volume. I'm working on migrating each server so the index files are stored on the normal drive primarily, then backed up after every index onto the EBS (and thus, the performance will improve). Currently, this change is only in place on the nuremberg server. For the record, the three keas apps are on the following servers:
keas-frontend-staging: cologne
keas-frontend-demo: donetsk
keas-frontend-production: nuremberg
So, your production server is using the new setup… but if that's not working very well for you, then I'm certainly keen to investigate further. How often are records updating? Are you using deltas or just relying on the attribute updates?
Cheers
Pat
3 Posted by Thomas Fisher on 27 Apr, 2012 08:17 PM
Hi Pat,
Thanks for your response. Both deltas and attribute updates are running. I
restricted deltas by overriding the indexed_data_changed? method (much of
the indexed data is based on SQL snippets, so I didn't want to trigger a
delta too often). I'm using a similar strategy for attribute updates.
We're getting airbrake errors for connection reset, tracing back to the
update attributes a few times a week.
Let me know if there's anything I can do to get you more information about
the problem.
Tommy