Failure to start: No such file or directory; NOT SERVING
I'm trying to get this set up on heroku. I have it working just fine locally.
The config loaded and the index created successfully. But when I try to start the service, it can't find my index file.
The service appears to start just fine:
listening on all interfaces, port=9313
but the "start" action fails because:
precaching index 'MyIndex'
[Wed Jun 26 19:03:32.984 2019] [ 9700] WARNING: index 'MyIndex': preload: failed to open /mnt/local/flying-sphinx/552dde8b36b080ec0/indexes/my-index.sph: No such file or directory; NOT SERVING
[Wed Jun 26 19:03:32.984 2019] [ 9700] FATAL: no valid indexes to serve
'
It looks like Flying Sphinx overwrites the paths to various files from my config file, which is fine, but it also means I don't know how to adjust for this problem. Any suggestions?
Keyboard shortcuts
Generic
? | Show this help |
---|---|
ESC | Blurs the current field |
Comment Form
r | Focus the comment reply box |
---|---|
^ + ↩ | Submit the comment |
You can use Command ⌘
instead of Control ^
on Mac
1 Posted by Sean Massa on 26 Jun, 2019 07:45 PM
It looks like trying again several times resolved the problem.
Support Staff 2 Posted by Pat Allan on 27 Jun, 2019 03:48 AM
Hi Sean,
Sorry that something didn’t quite go to plan the first time around - though it’s good to know things are working more smoothly now.
I’ll definitely have a look to try to make sure such issues don’t crop up again. Can you let me know which version/language of the flying-sphinx client library you’re using?
Thanks,
—
Pat
3 Posted by Sean Massa on 03 Jul, 2019 03:05 AM
Sure! I'm using Node.js libraries "flying-sphinx@0.2.3" and "sphinx-promise@1.0.5", which depends on "sphinxapi@1.3.0".
Support Staff 4 Posted by Pat Allan on 14 Jul, 2019 12:47 PM
Hi Sean,
Thanks for providing those details. The node client had been neglected for quite some time, but I’ve spent a good part of the last week updating it significantly. It now uses the v5 Flying Sphinx API (previously it’d been using v3), which is more reliable, using websockets to track action results (rather than polling, or just running long-lived requests).
These changes are released as v1.0.0 of the NPM package - if you have the chance to give it a shot, that’d be great! Do note the updated approach for managing configuration:
https://github.com/flying-sphinx/flying-sphinx-js#configuration <https://github.com/flying-sphinx/flying-sphinx-js#configuration>
(But the old approach should continue to work, and none of the public syntax has changed.)
One thing that’s now possible with the client updates and the configuration changes is the ability to specify the version of Sphinx you want to use. The old default was v2.0.6, which is quite ancient in Sphinx terms. I recommend updating to v2.2.11 (or even 3.1.1 - depending on what you use locally, of course!). Both of these versions should improve performance, and possibly reduce the Sphinx index file size. However: making this change will result in the need to rebuild your Sphinx setup (taking the daemon offline while the re-indexing happens), so definitely give this a spin on your staging environment first, and plan the production update for a time/situation where Sphinx being down will impact as few people as possible.
If you hit any issues with this new NPM package version, do let me know. I’ve tested it out (and it has a test suite covering all current Node versions), but definitely keen for feedback and feature suggestions from customers. Ruby is the language that has historically gotten the most love by Flying Sphinx customers, hence I’m especially keen on input from those using different languages.
Kind regards,
—
Pat
5 Posted by Sean Massa on 15 Jul, 2019 01:43 PM
Hey Pat! We'll check out the new version and let you know how it goes.
Right now, we're having the same issue again. Now that we're using this in production, our production searches are down. Can you look into this for us?
6 Posted by Sean Massa on 15 Jul, 2019 01:45 PM
Oh! I just saw that we received an error email, but I'm still not sure what to do about it.
Support Staff 7 Posted by Pat Allan on 15 Jul, 2019 01:57 PM
Hi Sean,
I’m just having a look now, but it might be that your disk usage is actually jumping up significantly and you’re maxing out the server’s disk space:
https://stackoverflow.com/questions/4367556/what-does-the-error-error-index-product-raw-hits-write-error-122070-of-26 <https://stackoverflow.com/questions/4367556/what-does-the-error-error-index-product-raw-hits-write-error-122070-of-26>
I can see that you’re running another indexing request now, so I’m keeping an eye on the disk space. There’s about 45GB free… so using all of that would be rather worrying - but you’re already using 20GB! I’m going to kill your existing indexer process, remove all existing index files (given they’re not usable at this point), just to ensure there’s as much disk space available as possible, and then run the indexer myself.
I’ll let you know how it goes.
—
Pat
Support Staff 8 Posted by Pat Allan on 15 Jul, 2019 02:03 PM
Just further to this, while the indexer is running again:
9 Posted by Sean Massa on 15 Jul, 2019 02:12 PM
Thanks for checking on this!
Sphinx Version: Yeah, if you think that'll help. I'm updating the node package locally now anyway.
Record Count: 1,236,233
Disk Space: We were including a large "description" column before, but have removed that. We did add more records since the last build, but I was hoping the column removal would make up for that in disk size. Let's see where we end up after this new clean run.
Support Staff 10 Posted by Pat Allan on 15 Jul, 2019 02:17 PM
Just killed my first manual attempt (using 2.0.6), switched to 2.2.11…
The 2.0.6 run had gotten to 324k records in 16 minutes. The 2.2.11 run has just hit the same mark in 4 minutes.
11 Posted by Sean Massa on 15 Jul, 2019 02:20 PM
Oh nice! Thanks for the update!
Does the new node package default to using Sphinx 2.2.11? That's what we use locally. I'm mostly wondering if I need to use the JavaScript configuration at all. If so, does it need to look like this?
I'll hold off on pushing anything, for sure. I'm just wondering for after we figure this out.
Support Staff 12 Posted by Pat Allan on 15 Jul, 2019 02:25 PM
The new v1.0.0 NPM package will not overwrite the version if you don't specify one (v0.2.3 will, though).
Granted, it doesn't hurt to have it set as part of your configuration process, but yeah, that's not something you need to do right now.
13 Posted by Sean Massa on 15 Jul, 2019 02:39 PM
I see search results again! I have a change ready on our end to upgrade the npm package and specify sphinx version 2.2.11. Should I deploy that?
Support Staff 14 Posted by Pat Allan on 15 Jul, 2019 02:40 PM
May have been a brief blip with the daemon just then - it had booted up with 2.0.6 thanks to monit, but is now 2.2.11. Feel free to deploy the new NPM package :)
Support Staff 15 Posted by Pat Allan on 15 Jul, 2019 02:40 PM
Disk usage is down to 1.4GB too! 🎉
Support Staff 16 Posted by Pat Allan on 15 Jul, 2019 02:42 PM
The output of the indexer, given that because it was a manual invocation it won't appear in your Flying Sphinx dashboard:
Took 25 minutes.
17 Posted by Sean Massa on 15 Jul, 2019 02:43 PM
Great! Thanks a bunch for your help.
Support Staff 18 Posted by Pat Allan on 15 Jul, 2019 02:45 PM
No worries at all, great to have everything upgraded and working faster and with lower disk usage. If there's any other queries or issues, do let me know. (Though right now it's approaching 1am here in Melbourne, so I'll likely be asleep for any immediate questions!)