Commit Graph

1745 Commits

Author SHA1 Message Date
Markus Heiser edfd0e2fe5 [fix] brave fetch_traits: Brave added Chinese (zh-hant) to UI
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-07-29 10:28:53 +02:00
Markus Heiser ee959ed9fc [fix] engine geizhals: if there are no offers, there is no best price
Fault pattern: if there are no offers, then an exception has been thrown:

    IndexError: list index out of range

This patch makes the addition of “best price” dependent on whether one exists.

Closes: #3685
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-07-28 19:00:51 +02:00
Bnyro 304ddd8114 [feat] videos template: support for view count 2024-07-27 11:49:58 +02:00
Bnyro 84abab0808 [feat] engine: implementation of geizhals.de 2024-07-27 11:46:25 +02:00
Sylvain Cau b9ddd59c5b [enh] Add API Key support for discourse.org forums 2024-07-27 09:21:40 +02:00
Markus Heiser 657dcb973a [fix] engine yacy: update list of base URLs
https://search.lomig.me
  Poor results / tested `!yacy :en hello` and got zero results

https://yacy.ecosys.eu
  Slow response (> 6sec for trivial search terms)

https://search.webproject.link
  Dead instance / URL offline

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-07-20 09:59:43 +02:00
Grant Lanham 9a4fa7cc4f Update mullvad_leta.py to account for img_elem
A recent update from Mullvad Leta introduced the img_elem. This update
broke the existing logic. Now, by checking the length of the dom_result
to see if it was included in the return results, we can handle the logic
accordingly.
2024-07-15 06:58:39 +02:00
Bnyro e4da22ee51 [feat] engine: implementation of alpine linux packages
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2024-07-14 17:57:58 +02:00
Grant Lanham ef103ba80a Implement google/brave switch in Mullvad Leta
cleanup

Import annontations
2024-07-07 08:08:11 +02:00
Bnyro 4eaa0dd275 [fix] gentoo: use mediawiki engine 2024-07-03 10:24:03 +02:00
Thomas Renard 39aaac40d6 [mod] libretranslate: add direct link to translation (engine) 2024-06-30 16:18:33 +02:00
Markus Heiser 0f9926b89a [fix] brave fetch_traits: layout of the settings page has changed
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-06-25 15:08:18 +02:00
Markus Heiser 39ffec87b7 [fix] engine zlibrary: handle seized domain
The domains of zlibrary instances are known to be seized from time to time.
This leads to problems when, for example, the automated tasks try to update the
engine traits (aka fetch_traits). The search function should also generate a
suitable error message (currently either SSL errors or empty result lists are
returned). [1]

[1] https://github.com/searxng/searxng/issues/3610
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-06-25 14:40:19 +02:00
Markus Heiser b8fa4d6195 [fix] bing news results return invalid images
Closes: https://github.com/searxng/searxng/issues/3502
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-06-25 11:12:41 +02:00
Grant Lanham 9a9ca307fe [fix] implement tests and remove usage of gen_useragent in engines 2024-06-23 11:51:41 +02:00
Richard Lyons f195d98bfb Fix search_url building. 2024-06-20 06:30:00 +02:00
Allen 13eec44b65 [fix] \!goi irrelevant results AND display more results 2024-06-16 16:45:03 +02:00
Bnyro e9f8412a6e [perf] torrents.html, files.html: don't parse and re-format filesize 2024-06-15 15:42:29 +02:00
Bnyro df15c21b35 [feat] mozhi: fix crash, support synonyms and definition 2024-06-15 11:33:09 +02:00
Bnyro 1fe13d0ba4 [refactor] duckduckgo: use extr helper function in get_vqd 2024-06-15 11:24:05 +02:00
Bnyro 46c5309888 [feat] mojeek: implement dedicated module 2024-06-07 11:31:05 +02:00
allendema_searxng_pi ee146dbc07 [enh] Add engine for discourse forums 2024-06-07 10:16:09 +02:00
Allen 0fa81fc782 [enh] add re-usable func to filter text 2024-05-29 17:56:17 +02:00
Jeff Alyanak 0fb3f0e4ae [fix] do not show DDG IP from zero click
The zero click result from DuckDuckGo for IP should not be displayed.  It will
return the IP of the searxng server, not the user's IP, and looks a bit strange
when the `self_info` plugin is enabled as two different IPs get returned.
2024-05-29 11:23:26 +02:00
Markus Heiser a20dfbbcbd [fix] engine startpage: fetch_traits() / if lang name unknown by babel
Workflow "Update data - update_engine_traits.py" fails last night [1].
This issue has already been reported by @allendema [2].

[1] https://github.com/searxng/searxng/actions/runs/9278028691/job/25528337485#step:6:168
[2] https://github.com/searxng/searxng/pull/3504/files#r1613559565

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-05-29 07:52:18 +02:00
Austin-Olacsi 9bb75a6644 [feat] engine: implementation of findthatmeme 2024-05-28 18:18:13 +02:00
Markus Heiser c19bffde4d [fix] issues reported by pylint-3.2.2
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-05-28 18:10:04 +02:00
Daniel Kukula 87165ac532 [mod] engine hex: add sort_criteria & page_size to configuration 2024-05-28 11:55:59 +02:00
allendema_searxng_pi 68365c8c1d [enh] add instant answers from ddg 2024-05-24 10:44:17 +02:00
Daniel Kukula a49232ee29 [feat] engine: implementation of cargo search (crates.io) 2024-05-17 16:37:39 +02:00
Markus Heiser 916739d6b4 [mod] simple theme: drop img_src from default results
The use of img_src AND thumbnail in the default results makes no sense (only a
thumbnail is needed).  In the current state this is rather confusing, because
img_src is displayed like a thumbnail (small) and thumbnail is displayed like an
image (large).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-05-16 07:30:38 +02:00
Bnyro 0f2f52f0b5 [fix] google: don't display that keyword is missing in content field 2024-05-15 16:03:35 +02:00
Markus Heiser 949a73103f [mod] hex engine: normalize (some of) the linked terms
The names of the links are rather tags than real names, and they sometimes vary
greatly in their spelling:

- GitHub: github, Github
- Source code: Repository, SCM, Project Source Code
- Documentation: docs, Documentation

It was standardized to terms such as 'Source code' and 'Documentation', as
translations already exist for these terms.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-05-15 12:50:35 +02:00
Daniel Kukula cc8b537e34 [mod] package.html template: additional links (a python dict)
- Closes: https://github.com/searxng/searxng/issues/3456
2024-05-15 12:50:35 +02:00
Bnyro 82b6c0d05f [feat] engine: implementation of gitea 2024-05-15 07:23:57 +02:00
Bnyro 60a373ad89 [fix] chefkoch: use German words in content field 2024-05-12 18:36:49 +02:00
Markus Heiser ffb1001f80 [fix] engine wikidata - WIKIDATA_UNITS has been changed in #3378
This patch is a leftover from [1] in which the WIKIDATA_UNITS values has become
a dictionary.

[1] https://github.com/searxng/searxng/pull/3378

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-05-10 16:04:27 +02:00
Markus Heiser fb32425d78 [mod] yacy engine: pick base_url randomly from a list of instances
Inspired by post [1] in the disscussion we had, while yacy.searchlab.eu was
broken.

[1] https://github.com/searxng/searxng/issues/3428#issuecomment-2101080101

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-05-09 17:29:15 +02:00
Markus Heiser dbed8da284 [fix] startpage engine: XPath expressions adapted for new HTML layout
Startpage has changed its HTML layout, classes like ``w-gl__result__main`` do no
longer exists and the result items have been slightly changed in their
structure.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-05-04 19:45:52 +02:00
Bnyro 78077126f2 [feat] wikimedia commons: support for videos, audio and other files 2024-05-04 06:23:04 +02:00
Daniel Kukula 46d7a8289b [feat] engine: implementation of https://hex.pm
The package manager for the Erlang ecosystem Find packages.

Co-authored-by: Bnyro <82752168+Bnyro@users.noreply.github.com>
2024-05-03 21:37:37 +02:00
Markus Heiser e6c345f890 [fix] ccc_media engine: filter video formats and ignore audio & SubRip
CCC media serves several recording formats, to name a few:

- application/x-subrip
- video/mp4
- video/webm
- audio/mpeg
- audio/opus
- audio/mpeg

not all of them are suitable for a video frame.  If available we should prefer
video/mp4 due to its minimal data rates.

Closes: https://github.com/searxng/searxng/issues/3431
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-04-28 08:42:18 +02:00
Bnyro 46efb2f36d [feat] plugins: new unit converter plugin 2024-04-27 18:11:33 +02:00
Bnyro 42b58eb448 [feat] media.ccc.de: implement module with pagination and iframe 2024-04-27 08:55:26 +02:00
Markus Heiser ddaa6ed759 [fix] add missing localizable (gettext) messages to searxng.msg
To test this patch I used .. and checked the diff of the `messages.pot` file::

    $ ./manage pyenv.cmd pybabel extract -F babel.cfg \
              -o ./searx/translations/messages.pot searx/
    $ git diff ./searx/translations/messages.pot

----

hint from @dalf: f-string are not supported [1] but there is no error [2].

[1] python-babel/babel#594
[2] python-babel/babel#715

Closes: https://github.com/searxng/searxng/issues/3412
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-04-26 07:34:32 +02:00
Bnyro 0a4280a137 [refactor] translation engines: add translate category
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2024-04-26 07:33:28 +02:00
Bnyro 91522f3801 [feat] engine: implementation of LibreTranslate
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2024-04-26 07:33:28 +02:00
Bnyro 644ae51417 [fix] yahoo: result titles are getting mixed together 2024-04-20 17:00:17 +02:00
singletail 1746eecf2b [fix] engine: youtube_api - ignore channels / prevent exceptions
`youtube_api.py` throws an exception if the search results contain a channel, as
channels have no videoId.  This PR adds a keycheck for parsing the json response.
2024-04-13 08:41:00 +02:00
Diagonalo 6458490271 [mod] engine 'mwmbl: Update API URL 2024-04-10 06:51:28 +02:00
Markus Heiser f1a148f53e [fix] ddg engine: if no vqd value can be determined, don't save None
Closes: https://github.com/searxng/searxng/issues/3370
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-04-08 10:56:11 +02:00
Austin-Olacsi b6fe2cad71 [mod] Ask engine: remove tracking paramaters and set max page to 5 2024-04-08 07:50:14 +02:00
Bnyro bf3845b093 [feat] engine: implementation of Open Meteo 2024-04-07 19:02:53 +02:00
Mikkel Denker b21aaa8907 Add user-agent to stract requests 2024-03-22 13:40:46 +01:00
Markus Heiser 542f7d0d7b [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION
In the past, some files were tested with the standard profile, others with a
profile in which most of the messages were switched off ... some files were not
checked at all.

- ``PYLINT_SEARXNG_DISABLE_OPTION`` has been abolished
- the distinction ``# lint: pylint`` is no longer necessary
- the pylint tasks have been reduced from three to two

  1. ./searx/engines -> lint engines with additional builtins
  2. ./searx ./searxng_extra ./tests -> lint all other python files

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-11 14:55:38 +01:00
Markus Heiser 8205f170ff [mod] pylint all engines without PYLINT_SEARXNG_DISABLE_OPTION
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-11 14:55:38 +01:00
Markus Heiser 707d6270c8 [doc] engine: mullvad leta
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-10 18:20:07 +01:00
Grant Lanham 9d6896a771 [feat] engine: implementation of mullvad leta 2024-03-10 18:20:07 +01:00
Markus Heiser 0ffec440b2 [doc] annas_archive: ctegs journal_article & book_any has been removed
In commit 8af181533 in PR:

- https://github.com/searxng/searxng/pull/3321

the category `journal_article` has been removed, `book_any` has been removed
longer time ago.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-10 12:46:00 +01:00
Markus Heiser e97e1f9110 [fix] duckduckgo.fetch_traist - URL of region definitions has changed
- https://duckduckgo.com/dist/util/u.7669f071a13a7daa57cb.js

updated from u661.js to u.7669f071a13a7daa57cb / should be updated
automatically?  The last change was on March 23rd in dba8977b09 [1]

- [1] https://github.com/searxng/searxng/pull/2269

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-10 10:32:54 +01:00
Markus Heiser 33c099022b [drop] engine framalibre - sarch URL is no longer supported
Closes: https://github.com/searxng/searxng/issues/3286
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-09 18:00:45 +01:00
Alexandre Flament 42515d98f7 engine npm use the packages.html template 2024-03-09 10:24:49 +01:00
Bnyro 6dd07b88a4 [feat] lib.rs: use packages.html template 2024-03-09 10:17:16 +01:00
Markus Heiser a7b51f023e [black] upgrade black 22.12.0 --> 24.2.0
The issue discussed in [1] has been solved since [2] has been merged into black
/ now we can upgrade without touching 69 files as it was needed with black
23.1.0 [3].

[1] https://github.com/searxng/searxng/pull/2159#issuecomment-1425723977
[2] https://github.com/psf/black/pull/4060
[3] https://github.com/searxng/searxng/pull/2159/files

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-09 08:15:50 +01:00
Bnyro 44decaa524 [feat] pkg.go.dev: use packages.html template 2024-03-08 17:18:54 +01:00
Markus Heiser a5b81e2555 [fix] nyaa engine - paging support & filesize (GiB)
BTW: pylint engine

Closes: https://github.com/searxng/searxng/issues/3290
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-07 07:38:31 +01:00
Markus Heiser 3529f0af50 [fix] engine searchcode.com - paging is broken in searchcode.com's API
paging is broken in searchcode.com's API .. not sure it will ever been fixed /
this commit disables paging in the engine and BTW pylint `searchcode_code.py`.

Closes: https://github.com/searxng/searxng/issues/3287
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-06 12:33:29 +01:00
Markus Heiser 5602f0bf96 [fix] engine pinterest - ignore result type 'story'
Close: https://github.com/searxng/searxng/issues/3289
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-06 07:33:43 +01:00
Markus Heiser d97b84bea2 [fix] ddg engines (get_vqd) - the vqd value is no longer in the form
Closes: https://github.com/searxng/searxng/issues/3276
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-05 16:27:04 +01:00
Yaksh Bariya b1431e1670 [feat] engine: implementation of cppreference 2024-03-03 17:07:29 +01:00
Bnyro f3b4bf86a7 [feat] engine: implementation of void linux packages
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2024-02-29 13:12:40 +01:00
Alexandre Flament d58760ef75 [mod] pypi engine: use packages.html 2024-02-29 07:48:44 +01:00
Austin-Olacsi 9330a072eb [feat] engine: implementation of pixiv 2024-02-25 16:29:57 +01:00
Bnyro e76ab1a4b3 [refactor] images: add resolution, image format and filesize fields
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2024-02-25 16:22:37 +01:00
Markus Heiser b683aa63fb [feat] github: use packages template
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-02-25 14:56:57 +01:00
Bnyro 1d6f475fbc [feat] dockerhub: use packages template 2024-02-25 14:56:57 +01:00
Bnyro cfbe59b6b3 [feat] engine: implementation of mozhi 2024-02-23 07:50:48 +01:00
Xinos 3c42252c78 [feat] engine: implementation of yandex music
Access to music is limited to a few countries:

- https://yandex.com/support/music/access.html
2024-02-21 17:17:12 +01:00
micsthepick fde21c8f99 [feat] engine: Add LiveSpace livestreams 2024-02-20 14:45:07 +01:00
Markus Heiser 76845ea42c [mod] engine ask.com - parse JS result to JSON
Parse the result list from ask.com given in the variable named
window.MESON.initialState::

    <script nonce="..">
        window.MESON = window.MESON || {};
        window.MESON.initialState = {"siteConfig": ...
          ...}};
        window.MESON.loadedLang = "en";
    </script>

The result list is in field::

    json_resp['search']['webResults']['results']

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-02-18 09:17:22 +01:00
Hackurei 3df53d6e50 [feat] engine: implementation of ask.com 2024-02-18 09:17:22 +01:00
Bnyro 11c0651ef4 [fix] lingva: redirect and parsing error 2024-02-11 11:28:47 +01:00
Hackurei b0fe0924a3 [feat] engine: implementation of brave goggles 2024-02-10 08:09:50 +01:00
Markus Heiser c14b117121 [fix] brave.search - update result selectors to fit to new layout
Closes: https://github.com/searxng/searxng/issues/3167
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-01-31 16:21:56 +01:00
Bnyro efbee96b1d [feat] brave: fix empty content and parse published dates 2024-01-30 07:41:45 +01:00
Bnyro 559b860618 [feat] engine: implementation of goodreads 2024-01-29 16:07:24 +01:00
Bnyro 8c73aa772b [fix] semantic scholar: bad request 2024-01-25 06:12:08 +01:00
Jinyuan Huang 9c5f9735cf [fix] engine: safesearch parameter in Google Videos engine (#2762)
Closes: https://github.com/searxng/searxng/issues/2762
2024-01-16 15:29:09 +01:00
Markus Heiser e560d7e373 [mod] presearch: add language & region support
In Presearch there are languages for the UI and regions for narrowing down the
search.  With this change the SearXNG engine supports a search by region.  The
details can be found in the documentation of the source code.

To test, you can search terms like::

   !presearch bmw :zh-TW
   !presearch bmw :en-CA

1. You should get results corresponding to the region (Taiwan, Canada)
2. and in the language (Chinese, Englisch).
3. The context in info box content is in the same language.

Exceptions:

1. Region or language is not supported by Presearch or
2. SearXNG user did not selected a region tag, example::

    !presearch bmw :en

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-01-15 19:23:26 +01:00
Markus Heiser a2c269bbac [mod] presearch: hardening engine's response against KeyErrors
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-01-15 19:23:26 +01:00
Bnyro 0a78f59aba [fix] presearch: safesearch, time ranges, crash when no result found 2024-01-15 19:23:26 +01:00
Bnyro 621e1313af [feat] engine: implementation of stract (beta) 2024-01-07 11:18:16 +01:00
Markus Heiser 8dfc1dbc5c [fix] engine - offical URL moved to https://lingva.thedaviddelta.com/
The ML top-level domain has been removed from Freenom ... moved the official
instance to https://lingva.thedaviddelta.com  [1]

To test use translation syntax:

    !lv en-de Alternative front-end for Google Translate

[1] https://fosstodon.org/@thedaviddelta/111376540936289966

Closes: https://github.com/searxng/searxng/issues/2718
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-12-25 09:57:35 +01:00
Markus Heiser 6df922b9da [fix] fetch_traits of google and zlibrary
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-12-24 08:28:32 +01:00
Bnyro 8a4104b992 [feat] engine: implementation of rotten tomatoes 2023-12-11 18:59:36 +01:00
Bnyro dda74fc9c9 [feat] engine: implementation of mdn 2023-12-11 18:57:40 +01:00
Bnyro b5b81c9806 [feat] engine: implementation of fyyd 2023-12-11 18:56:32 +01:00
Bnyro 527e13ab46 [feat] engine: implementation of podcastindex.org 2023-12-11 18:55:16 +01:00
Bnyro 3bc85c511c [mod] tootfinder: implement python module 2023-12-11 18:54:24 +01:00
Markus Heiser 3829c253ff [mod] add option max_page to bing, brave, qwant, startpage & mojeek
[1] https://github.com/searxng/searxng/issues/2982#issuecomment-1808975780

Reported-by: @Damaj301damaj-lol [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-12-03 13:47:17 +01:00