Skip to main content


 

Friendica search is working very slow, even timing out with a 504


@Hypolite Petovan , maybe you could shed some light on how to better deal with the search feature... Almost every search I perform results in a 504 error, at least the first few times I try. I've increased the number of php fpm workers and lowered the nice value a bit (also for nginx workers), as my server does do quite a few tasks - but it's pretty finely tuned and resources are typically available (unless things get really hectic). Is there anything else I can do? I've tested under my lightest loads, even after freshly restarting php & nginx. Everything else on my site is fairly low latency. I see from the admin settings that the search does a global search in the background to cache it for later searches, but it seems like it doesn't work until I finally get it to not timeout with the 504...

Could this have more to do with the remote server than my own? Now, I know a 504 is a gateway timeout and means the remote server didn't give us everything we needed in... show more

The trace you got is invaluable data when troubleshooting slowness. The problem seems to stem from the fact that your node didn't know part of all the up-to-80 contacts that the global directory sent based on your search term. Which means that for each unknown contact, your node will try to import it, which involves several curl requests to the remote server for network/protocol detection for example. The time spent sending these requests sequentially adds up to the search page processing time, and it seems PHP-FPM has a timeout set at 75 seconds.

I see two solutions to this problem:
  • Retry the search until all the remote contacts comprising the result have been imported in your local node, and the search should become faster.
  • Ditch the local contact import and use the directory results as-is. I know it will sadden @Rabuzarus because this means less magic authentication links, but I'm not sure it's relevant in this context.

Thanks @Hypolite Petovan !

I tried to find the way to ‘ditch the local contact import’, and the setting I disabled first was ‘search the local directory’, which does solve the problem of timing out, but it displays no global results at all now, ha! Since you said it would use the results as-is I guess this was the incorrect setting.

Is the setting you’re referring to something I need to put in the local config manually? I’m not at my workstation and don’t remember the exact file file name, but I believe it’s in the config dir in Friendica’s root... maybe /config/local.config.php?

Thanks again, the help is greatly appreciated!!

Oh, I'm sorry, I was talking about changing the code itself not to import the remote profiles, not about some setting you could change.

Oh I see. Hmm, would that be something fairly trivial, like a few edits in a file (it’s been about a decade since I’ve done any serious PHP stuff haha) or more of a can of worms sort of change?? Haha. Maybe if it’s more the former you could point me to a file to have a go at it?

Maybe as a backup plan, would lowering the queried profiles to something smaller than 80 be a reasonable temporary work-around?

Yes, you can lower it to 30 for example in mod/dirfind.php line 106.

I tried setting it to 30 and did quite a few searches over the last 30 minutes. It timed out about 50% of the time with this setting, and always took ~1min when it did work. That was an improvement, of course... but I lowered it to 20, expecting only marginally better results. However, to my surprise now any search I do populates results in a second or two. I’m not repeating queries, to rule out caching, and I am searching common things that generate many pages of results. The same positive results when I click a page number too, it’s lightning fast (relatively speaking, from where we were! Ha).

I don’t understand this behavior, but I’m very happy with it! I was I was under the impression setting this to 20 would result in only 20 results appearing, at most, but apparently I haven’t fully grok’d it yet ha.

Thank you so much @Hypolite Petovan & @Tobias ! You guys are amazing! #friendica forever! :P

I think I spoke to soon here... I guess maybe, despite my best efforts, I had searched for something that had been searched maybe, I don’t know. But I’m down to 10 and it still timed out. But I think I see some other things to play with in that file, I’m going to get back on this issue tomorrow. Thanks for pointing me in the right direction though!

Have you activated the option to only search in the tags? It's a setting in the admin panle. On some nodes the full-text search that is used instead leads to time-outs.

No, it isn't the item search, this is the official directory profile search.

Then there is the option to turn on the local search and perform a discovery of global contacts. Every search will also be done against the global directory, but by a worker process, so that the results should improve over time.

I’ll give that a try next! Thank you!