Implementations of the *traits* of the engines.
Engine's traits are fetched from the origin engine and stored in a JSON file in
the *data folder*. Most often traits are languages and region codes and their
mapping from SearXNG's representation to the representation in the origin search
engine.
To load traits from the persistence::
searx.enginelib.traits.EngineTraitsMap.from_data()
For new traits new properties can be added to the class::
searx.enginelib.traits.EngineTraits
.. hint::
Implementation is downward compatible to the deprecated *supported_languages
method* from the vintage implementation.
The vintage code is tagged as *deprecated* an can be removed when all engines
has been ported to the *traits method*.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
settings.yml:
* The default URL was unix:///usr/local/searxng-redis/run/redis.sock?db=0
* The default URL is now "false"
The default URL makes the log difficult to deal with:
if the admin didn't install a Redis instance, the logs record a false error.
It worked before because SearXNG initialized the Redis connection when the limiter started.
In this commit, SearXNG initializes Redis in searx/webapp.py
so various components can use Redis without taking care of the initialization step.
Most engines that support languages (and regions) use the Accept-Language from
the WEB browser to build a response that fits to the language (and region).
- add new engine option: send_accept_language_header
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
With Sphinx-doc update 4.4.0 we get some warnings about links that can be
replaced by already defined 'sphinx.ext.extlinks':
admin/engines/sql-engines.rst:144: WARNING: hardcoded link 'https://pypi.org/project/mysql-connector-python' could be replaced by an extlink (try using ':pypi:`mysql-connector-python`' instead)
docs/admin/installation-switch2ng.rst:10: WARNING: hardcoded link 'https://github.com/searxng/searxng/pull/446#issuecomment-954730358' could be replaced by an extlink (try using ':pull:`446#issuecomment-954730358`' instead)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The ? search operator has been broken for some time and
currently only raises the question why it's still there.
## Context ##
The query "Paris !images" searches for "Paris" in the "images" category.
Once upon a time Searx supported "Paris ?images" to search for "Paris"
in the currently enabled categories and the "images" category.
The feature makes sense ... the ? syntax does not.
We will hopefully introduce a +!images syntax in the future.
Fixes#702.
* allow not to record metrics (response time, etc...)
* this commit doesn't change the UI. If the metrics are disabled
/stats and /stats/errors will return empty response.
in /preferences, the columns response time and reliability will be empty.
The tab icon names are currently hard coded in the templates.
This commit lets us introduce an icon property in the future, e.g:
categories_as_tabs:
general:
icon: search-outline
Add a redis connector, the default DB connector is a socket at::
unix:///usr/local/searxng-redis/run/redis.sock?db=0
To set up a redis instance simply use::
$ ./manage redis.build
$ sudo -H ./manage redis.install
A hint for developers:
To get access rights to this instance, your developer account needs to be added
to the *searxng-redis* group::
$ sudo -H ./manage redis.addgrp "${USER}"
# don't forget to logout & login to get member of group
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Previously all categories were displayed as search engine tabs.
This commit changes that so that only the categories listed under
categories_as_tabs in settings.yml are displayed.
This lets us introduce more categories without cluttering up the UI.
Categories not displayed as tabs can still be searched with !bangs.
Previously the documentation grouped the engines by their first
category so e.g. YouTube and Invidious were only shown in the
in the videos section but not in the music section.
This commit fixes this by iterating over searx.engines.categories,
which also has the added benefit that the sections are now in the
same order as the tabs in the user interface.
In commit 4b43775c9 the brand.git_url & brand.git_branch has been removed, with
this patch these settings are removed from the documentation.
docs/admin/engines/settings.rst
- Remove brand.git_url & brand.git_branch
docs/dev/makefile.rst
- Remove brand.git_url & brand.git_branch
modified docs/dev/lxcdev.rst
- Remove brand.git_url & brand.git_branch
- fix searXNG to SearXNG
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Not all settings from the 'brand:' section of the YAML files are needed in the
shell scripts. This patch reduce the variables in ./utils/brand.env to what is
needed. The following ('brand:' settings) can be removed from this file:
- ISSUE_URL
- DOCS_URL
- PUBLIC_INSTANCES
- WIKI_URL
Tasks running outside of an *installed instance*, need the following settings
from the YAML configuration:
- GIT_URL <--> brand.git_url
- GIT_BRANCH <--> brand.git_branch
- SEARX_URL <--> server.base_url (aka PUBLIC_URL)
- SEARX_PORT <--> server.port
- SEARX_BIND_ADDRESS <--> server.bind_address
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In commit 94851790 we have centralized all SearXNG setups in the settings.yml
file:
94851790 [mod] move brand options from Makefile to settings.yml
This step has not yet been completed for the installation procedures! Since all
SearXNG setups are done in the settings.yml these environment variables needs to
be removed from the ./conf.sh file. Scripts and other tasks running outside of
an instance got the needed values from the ./utils/brand.env file.
By example: ATM the environment variables of the ./config.sh file are in
conflict with them from settings.yml:
- PUBLIC_URL --> {server:base_url}
- SEARX_INTERNAL_HTTP --> {server:bind_address}.{server:port}
- GIT_BRANCH --> {brand:GIT_URL}
These environment variable of a SearXNG instance and additional
- SEARX_SETTINGS_TEMPLATE
has been remove from the '.config.sh' file. With this patch, the main focus of
./conf.sh resists on environment variables needed for the installation of morty,
filtron software.
modified .config.sh:
- removed no longer supported variables (see above)
- add comment about: SearXNG setup in settings.yml
modified utils/searx.sh:
- SEARX_INTERNAL_HTTP no longer take from .config.sh
- SEARX_SETTINGS_PATH /etc/searx/settings.yml
- SEARX_SETTINGS_TEMPLATE obsolete
modified utils/lib_install.sh:
Initialize environment variables SEARX_PYENV, SEARX_SETTINGS_PATH and
PUBLIC_URL.
modified: utils/morty.sh
Add missing hint about SEARX_SETTINGS_PATH and move PUBLIC_URL to
utils/lib_install.sh
modified: utils/morty.sh
Move PUBLIC_URL to utils/lib_install.sh
Renamed utils/templates/etc/searx/use_default_settings.yml -> settings.yml
- removed option which can't be modified after installation
- add some comments with examples
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
modified docs/admin/engines/settings.rst
- Fix documentation and add section 'brand'.
- Add remarks about **buildenv** variables.
- Add remarks about settings from environment variables $SEARX_DEBUG,
$SEARX_PORT, $SEARX_BIND_ADDRESS and $SEARX_SECRET
modified docs/admin/installation-searx.rst & docs/build-templates/searx.rst
Fix template location /templates/etc/searx/settings.yml
modified docs/dev/makefile.rst
Add description of the 'make buildenv' target and describe
- we have all SearXNG setups are centralized in the settings.yml file
- why some tasks need a utils/brand.env (aka instance's buildenv)
modified manage
Settings file from repository's working tree are used by default and
ask user if a /etc/searx/settings.yml file exists.
modified searx/settings.yml
Add comments about when it is needed to run 'make buildenv'
modified searx/settings_defaults.py
Default for server:port is taken from enviroment variable SEARX_PORT.
modified utils/build_env.py
- Some defaults in the settings.yml are taken from the environment,
e.g. SEARX_BIND_ADDRESS (searx.settings_defaults.SHEMA). When the
'brand.env' file is created these enviroment variables should be
unset first.
- The CONTACT_URL enviroment is not needed in the utils/brand.env
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The blog article 'Query your local search engines' has been renamed 'Local
Search Engines', revised and moved into admin's chapter 'Engine & Settings'.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch is a marginal revision of the article "settings.yml", most changes
are from normalizing the YAML syntax.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- Split chapter "Engines" and rename it into "Engines & Settings"
- Move docs/admin/engines.rst -> docs/admin/engines/engine_settings.rst
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The article "Command line engines" should be in admin's engine
documentation (like the recoll engine).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- add sidebars with addition infos about commands and docker in general
- fix long lines & indentation
- correct link to https://github.com/searxng/searxng-docker
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Add script docker-entrypoint.sh to shellcheck and try to _simplify_ and
_normalize_ some parts:
- fix issues reported by shellcheck
- don't mix tab and space indent
- command 'help' replaced by '-h': ./dockerfiles/docker-entrypoint.sh -h
- replace printf in help() by 'cat <<EOF'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
make docker.buildx : build and push multiarch build.
(it can't be only build)
use buildx with the --cache-from and --cache-to options to cache the layers
(only the last built is cached)
settings.yml:
* outgoing.networks:
* can contains network definition
* propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
* retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
* local_addresses can be "192.168.0.1/24" (it supports IPv6)
* support_ipv4 & support_ipv6: both True by default
see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
* either a full network description
* either reference an existing network
* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
Markus Heiser
blob42
Alexandre Flament
Alexandre FLAMENT
Léon Tiekötter
Brock Vojković
samsaptidev
Martin Fischer
Fabio Berchtold
Noémi Ványi
ColonisationCaptain
Alex Balgavy