Commit Graph

482 Commits

Author SHA1 Message Date
return42 b5009b8610 [data] update searx.data - update_engine_descriptions.py 2024-09-29 13:17:43 +02:00
return42 d6b04d3ba1 [data] update searx.data - update_engine_traits.py 2024-09-29 13:17:06 +02:00
return42 bc6ee05422 [data] update searx.data - update_currencies.py 2024-09-29 13:16:45 +02:00
return42 a4558dda47 [data] update searx.data - update_firefox_version.py 2024-09-29 13:16:25 +02:00
return42 eb31eaaba0 [data] update searx.data - update_ahmia_blacklist.py 2024-09-29 13:15:57 +02:00
return42 609ead9ffe [data] update searx.data - update_wikidata_units.py 2024-09-29 13:14:53 +02:00
Markus 55e2f4a97f [data] update searx.data - update_engine_traits.py 2024-09-15 12:48:35 +02:00
return42 526428a69b [data] update searx.data - update_ahmia_blacklist.py 2024-08-29 17:49:14 +02:00
return42 d6ee8f38dd [data] update searx.data - update_wikidata_units.py 2024-08-29 17:48:59 +02:00
return42 d72fbcfd46 [data] update searx.data - update_firefox_version.py 2024-08-29 17:48:25 +02:00
return42 f44775c05a [data] update searx.data - update_engine_descriptions.py 2024-08-29 17:48:09 +02:00
return42 71451e5770 [data] update searx.data - update_currencies.py 2024-08-29 17:47:47 +02:00
return42 e76a4f72ef [data] update searx.data - update_external_bangs.py 2024-08-02 10:55:07 +02:00
Markus Heiser 98c73010f1 [data] update searx.data - update_engine_traits.py
$ make data.traits

Last GH action has been failed [1], the bugfixes from aa05685cc
were necessary to update the data.

[1] https://github.com/searxng/searxng/actions/runs/10135834050/job/28023757191

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-07-29 10:28:53 +02:00
return42 3196e7e86b [data] update searx.data - update_currencies.py 2024-07-29 07:02:52 +02:00
return42 7d47c961c3 [data] update searx.data - update_engine_descriptions.py 2024-07-29 07:01:39 +02:00
return42 ac51c77c33 [data] update searx.data - update_wikidata_units.py 2024-07-29 07:01:01 +02:00
return42 5cba412784 [data] update searx.data - update_ahmia_blacklist.py 2024-07-29 07:00:14 +02:00
return42 fff7792e32 [data] update searx.data - update_firefox_version.py 2024-07-29 07:00:00 +02:00
return42 adaec68728 [data] update searx.data - update_wikidata_units.py 2024-06-29 07:20:59 +02:00
return42 47ffa711d2 [data] update searx.data - update_ahmia_blacklist.py 2024-06-29 07:10:59 +02:00
return42 c7d1f4278d [data] update searx.data - update_firefox_version.py 2024-06-29 07:09:16 +02:00
return42 985c8b0bce [data] update searx.data - update_engine_traits.py 2024-06-29 07:08:55 +02:00
return42 7200640055 [data] update searx.data - update_engine_descriptions.py 2024-06-29 07:07:32 +02:00
Markus Heiser 1449824165 [data] update searx.data - update_engine_traits.py
$ make data.traits

Last GH action has been failed [1], the bugfixes from

- https://github.com/searxng/searxng/pull/3611
- https://github.com/searxng/searxng/pull/3612

were necessary to update the data.

[1] https://github.com/searxng/searxng/actions/runs/9278028691/job/25528337485

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-06-25 15:20:51 +02:00
Markus Heiser bb7c5bd44c [data] update searx.data - update_engine_traits.py
$ make data.traits

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-05-29 07:52:18 +02:00
return42 8713aa6c4b [data] update searx.data - update_wikidata_units.py 2024-05-29 06:54:24 +02:00
return42 c16a143239 [data] update searx.data - update_currencies.py 2024-05-29 06:53:45 +02:00
return42 8a78d7b64d [data] update searx.data - update_firefox_version.py 2024-05-29 06:53:14 +02:00
return42 f2763e5efc [data] update searx.data - update_ahmia_blacklist.py 2024-05-29 06:52:39 +02:00
return42 8250773ace [data] update searx.data - update_engine_descriptions.py 2024-05-29 06:52:17 +02:00
Markus Heiser 11fe88bb40 [fix] update wikidata units - remove URL prefix from Q-name
Sometimes the URL prefix switches from a http to a https, this patch harden the
code that removes the URL prefix from wikidata Q-name, issue has been reported
in [1].

[1] https://github.com/searxng/searxng/pull/3437#issuecomment-2082121730

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-05-01 18:58:28 +02:00
return42 e45a7cc063 [data] update searx.data - update_engine_traits.py 2024-04-29 10:02:48 +02:00
return42 c0b6c26eea [data] update searx.data - update_firefox_version.py 2024-04-29 10:02:16 +02:00
return42 0ec406cb7e [data] update searx.data - update_ahmia_blacklist.py 2024-04-29 10:01:52 +02:00
return42 ea585d4e77 [data] update searx.data - update_currencies.py 2024-04-29 09:48:56 +02:00
return42 d4e98c03a0 [data] update searx.data - update_engine_descriptions.py 2024-04-29 09:46:26 +02:00
Bnyro b3b1258e4e [data] wikidata: update wikidata_units.json 2024-04-27 18:11:33 +02:00
return42 ad9537065a [data] update searx.data - update_firefox_version.py 2024-03-29 08:44:18 +01:00
return42 531942ed16 [data] update searx.data - update_ahmia_blacklist.py 2024-03-29 08:43:50 +01:00
return42 433cfd868a [data] update searx.data - update_currencies.py 2024-03-29 08:42:16 +01:00
return42 22d68aa2a7 [data] update searx.data - update_engine_traits.py 2024-03-29 08:41:44 +01:00
return42 cc124b8f14 [data] update searx.data - update_engine_descriptions.py 2024-03-29 08:41:00 +01:00
Markus Heiser 542f7d0d7b [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION
In the past, some files were tested with the standard profile, others with a
profile in which most of the messages were switched off ... some files were not
checked at all.

- ``PYLINT_SEARXNG_DISABLE_OPTION`` has been abolished
- the distinction ``# lint: pylint`` is no longer necessary
- the pylint tasks have been reduced from three to two

  1. ./searx/engines -> lint engines with additional builtins
  2. ./searx ./searxng_extra ./tests -> lint all other python files

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-11 14:55:38 +01:00
Markus Heiser 3edaa26dae [data] make data.all
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-10 15:56:50 +01:00
return42 33b4ffee29 [data] update searx.data - update_engine_traits.py 2024-03-10 12:46:00 +01:00
return42 a5d36211a3 [data] update searx.data - update_ahmia_blacklist.py 2024-03-10 12:45:15 +01:00
return42 2cb389e9ce [data] update searx.data - update_currencies.py 2024-03-10 12:44:44 +01:00
return42 4a8867e646 [data] update searx.data - update_engine_descriptions.py 2024-03-10 12:01:33 +01:00
return42 7bbcf1be9a [data] update searx.data - update_external_bangs.py 2024-03-10 12:01:00 +01:00
dalf 35873b5a1c Update searx.data - update_ahmia_blacklist.py 2024-02-29 07:47:54 +01:00
dalf df74b5b59b Update searx.data - update_firefox_version.py 2024-02-29 07:47:01 +01:00
dalf 473b2dd42f Update searx.data - update_wikidata_units.py 2024-02-29 07:46:27 +01:00
dalf 9a474715e8 Update searx.data - update_engine_descriptions.py 2024-02-29 07:46:02 +01:00
Markus Heiser 894f164869 [fix] sort RTL_LOCALES before written into locales.json
To avoid unnecessary changes to the file, the list should be sorted before it is
written to the file.

You can test it by calling multiple times::

    make data.locales

and searx/data/locales.json should be unchanged.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-02-20 12:29:13 +01:00
Alexandre Flament ed66ed758d [mod] reduce memory footprint by not calling babel.Locale.parse at runtime
babel.Locale.parse loads more than 60MB in RAM.  The only purpose is to get:

    LOCALE_NAMES   - searx.data.LOCALES["LOCALE_NAMES"]
    RTL_LOCALES    - searx.data.LOCALES["RTL_LOCALES"]

This commit calls babel.Locale.parse when the translations are update from
weblate and stored in::

    searx/data/locales.json

This file can be build by::

    ./manage data.locales

By store these variables in searx.data when the translations are updated we save
round about 65MB (usually 4 worker = 260MB of RAM saved.

Suggested-by: https://github.com/searxng/searxng/discussions/2633#discussioncomment-8490494
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2024-02-20 10:43:20 +01:00
dalf bffc4905ce Update searx.data - update_currencies.py 2024-02-10 23:55:41 +01:00
dalf 62fa85287a Update searx.data - update_wikidata_units.py 2024-01-29 14:04:24 +01:00
dalf 0ed7548d83 Update searx.data - update_firefox_version.py 2024-01-29 14:04:03 +01:00
dalf d763934a7a Update searx.data - update_ahmia_blacklist.py 2024-01-29 14:03:03 +01:00
dalf 14f73ef3d9 Update searx.data - update_engine_traits.py 2024-01-29 14:02:30 +01:00
dalf a393341403 Update searx.data - update_engine_descriptions.py 2024-01-29 13:34:12 +01:00
dalf 4647981419 Update searx.data - update_wikidata_units.py 2023-12-29 06:09:04 +01:00
dalf a8cc101c35 Update searx.data - update_ahmia_blacklist.py 2023-12-29 06:08:35 +01:00
dalf 59a67f30c8 Update searx.data - update_firefox_version.py 2023-12-29 06:07:46 +01:00
dalf 81e8638de0 Update searx.data - update_currencies.py 2023-12-29 06:06:51 +01:00
Markus Heiser 2863b0f6da Update searx.data - update_engine_traits.py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-12-24 08:28:32 +01:00
Markus Heiser 3665b32aff Revert "[fix] update user agent"
This reverts commit 3c6549a17f.

Related:

- https://github.com/searxng/searxng/pull/2826
2023-12-23 07:48:38 +01:00
dalf 5a72977efa Update searx.data - update_currencies.py 2023-11-29 06:34:25 +01:00
dalf b18bef17a7 Update searx.data - update_engine_descriptions.py 2023-11-29 06:24:03 +01:00
dalf a136b0b44d Update searx.data - update_ahmia_blacklist.py 2023-11-29 06:18:21 +01:00
dalf 3ca84cb8d7 Update searx.data - update_wikidata_units.py 2023-11-29 06:17:32 +01:00
dalf b63841f0a1 Update searx.data - update_engine_traits.py 2023-10-29 09:04:24 +01:00
dalf a9d778ee0f Update searx.data - update_wikidata_units.py 2023-10-29 08:45:15 +01:00
dalf 92efb941d8 Update searx.data - update_firefox_version.py 2023-10-29 08:44:59 +01:00
dalf fa494b9be3 Update searx.data - update_currencies.py 2023-10-29 08:44:35 +01:00
dalf 1b92ea4e70 Update searx.data - update_engine_descriptions.py 2023-10-29 08:44:17 +01:00
dalf 55e7cb3e33 Update searx.data - update_ahmia_blacklist.py 2023-10-29 08:17:29 +01:00
Bnyro 48cb58bd2e [feat] duckduckgo: support for videos and news 2023-10-09 06:53:43 +02:00
Markus Heiser fd1422a670 [mod] engine - simplify region & lang handling, make filters configurable
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-05 10:55:08 +02:00
Markus Heiser 32a4ea350e [fix] Revision of the Bing engines
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-01 08:01:38 +02:00
dalf ae2762eb78 Update searx.data - update_firefox_version.py 2023-09-29 06:58:11 +02:00
dalf 18da3d65c5 Update searx.data - update_ahmia_blacklist.py 2023-09-29 06:57:40 +02:00
dalf 4a66fa3e7d Update searx.data - update_wikidata_units.py 2023-09-29 06:56:50 +02:00
dalf 9be6e20256 Update searx.data - update_engine_traits.py 2023-09-29 06:56:10 +02:00
dalf d821ec44dd Update searx.data - update_currencies.py 2023-09-29 06:55:04 +02:00
dalf cdee5aa3b9 Update searx.data - update_engine_descriptions.py 2023-09-29 06:53:05 +02:00
Markus Heiser 0c39064d60 Update searx.data - update_engine_traits.py
Update searx.data - update_engine_traits.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-27 09:12:38 +02:00
Markus Heiser 3a456b1282 [fix] engine annas archive - fetch traits (modified xpath selectors)
Anna’s Archive has cleaned up their languages, available file extensions and
changed the HTML form.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-27 09:12:38 +02:00
jazzzooo 3c6549a17f [fix] update user agent 2023-09-25 22:46:22 +02:00
Bnyro bf7af99ef2 [mod] make data.traits: add Odysee 2023-09-13 21:41:33 +02:00
dalf f57842b05d Update searx.data - update_firefox_version.py 2023-08-29 07:21:46 +02:00
dalf a5cd777350 Update searx.data - update_ahmia_blacklist.py 2023-08-29 07:21:21 +02:00
dalf f361a10af6 Update searx.data - update_currencies.py 2023-08-29 07:20:42 +02:00
dalf 978062bdbe Update searx.data - update_engine_descriptions.py 2023-08-29 07:20:06 +02:00
dalf f59fd8dc9b Update searx.data - update_wikidata_units.py 2023-08-29 07:17:44 +02:00
Markus Heiser b8352eca0c [mod] brave engines: add fetch_traits() / improve language support
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-08 16:21:45 +02:00
dalf 8b4ba204b7 Update searx.data - update_engine_descriptions.py 2023-07-29 11:02:54 +02:00
dalf d421385dc6 Update searx.data - update_wikidata_units.py 2023-07-29 11:02:23 +02:00
dalf 290f16207b Update searx.data - update_currencies.py 2023-07-29 11:02:14 +02:00