{"id":55174,"date":"2024-08-02T16:23:28","date_gmt":"2024-08-02T15:23:28","guid":{"rendered":"https:\/\/proxidize.com\/?post_type=use-cases&#038;p=55174"},"modified":"2025-10-02T12:12:16","modified_gmt":"2025-10-02T11:12:16","slug":"parsing-html-python-pyquery","status":"publish","type":"blog","link":"https:\/\/proxidize.com\/blog\/parsing-html-python-pyquery\/","title":{"rendered":"Guide to PyQuery: Parsing HTML with Python"},"content":{"rendered":"\n<p>An essential part of web development and data extraction is parsing HTML with Python, amongst many other coding languages. One tool that can assist with this is PyQuery. It is a popular choice for parsing HTML as it simplifies the process with its jQuery-like syntax which makes it more accessible. When compared to other powerful libraries, PyQuery offers a unique blend of simplicity and functionality.<\/p>\n\n\n\n<p>This guide to <a href=\"https:\/\/pyquery.readthedocs.io\/\" data-type=\"link\" data-id=\"https:\/\/pyquery.readthedocs.io\/\" target=\"_blank\" rel=\"noopener\">PyQuery<\/a> will explore parsing HTML with Python. From basic setup and element selection to more advanced features such as DOM manipulation, you will learn how to navigate and modify HTML documents with ease.<\/p>\n\n\n\n<div style=\"height:24px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized centered\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/introduction-to-pyquery-1024x576.png\" alt=\"An animated laptop with the title &quot;Introduction to PyQuery&quot;.\" class=\"wp-image-55197\" style=\"object-fit:cover\" srcset=\"https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/introduction-to-pyquery-1024x576.png 1024w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/introduction-to-pyquery-300x169.png 300w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/introduction-to-pyquery-768x432.png 768w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/introduction-to-pyquery-1536x864.png 1536w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/introduction-to-pyquery-600x338.png 600w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/introduction-to-pyquery.png 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<div style=\"height:24px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Introduction to PyQuery<\/h2>\n\n\n\n<p>PyQuery is a Python library that is designed to make working with HTML documents easier. It allows you to use familiar CSS syntax to select, navigate from one element to another, and manipulate HTML elements. This makes it an excellent choice for developers who are used to the <a href=\"https:\/\/jquery.com\/\" target=\"_blank\" rel=\"noopener\">jQuery library<\/a> in Javascript and want a similar experience in Python.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Comparison With Other HTML Parsing Libraries<\/h3>\n\n\n\n<p>There are several popular libraries to choose from that can assist with parsing HTML in Python. Two of these libraries include Beautiful Soup and lxml.<\/p>\n\n\n\n<p><a href=\"https:\/\/proxidize.com\/use-cases\/web-scraping-with-beautiful-soup\/\">Beautiful Soup<\/a> provides a simple API for navigating and searching the parse document tree. Lxml is fast and efficient and leverages the power of the libxml2 and <a href=\"http:\/\/xmlsoft.org\/libxslt\/index.html\" target=\"_blank\" rel=\"noopener\">libxslt libraries<\/a>. It offers both an ElementTree API and a BeautifulSoup-like API for parsing and manipulating HTML. However, PyQuery stands out by combining the ease of use of BeautifulSoup with the speed of lxml.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Key Features and Advantages of using PyQuery<\/h3>\n\n\n\n<p>PyQuery offers many features and advantages that make it a powerful tool for HTML parsing including:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>CSS Selectors: PyQuery&#8217;s selectors allow you to quickly and easily locate elements within an HTML document.<\/li>\n\n\n\n<li>Ease of Use: The syntax and API are straightforward and intuitive.<\/li>\n\n\n\n<li>Integration with lxml: By leveraging lxml, PyQuery provides fast and efficient parsing and manipulation of HTML documents.<\/li>\n\n\n\n<li>DOM Traversal and Manipulation: PyQuery allows for comprehensive <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/API\/Document_Object_Model\" target=\"_blank\" rel=\"noopener\">Document Object Model<\/a> (DOM) traversal and manipulation, including adding, removing, and modifying elements.<\/li>\n\n\n\n<li>Compatibility with jQuery Code: If you have existing jQuery code, you can often translate it directly to PyQuery with minimal changes, facilitating a smooth transition between JavaScript and Python.<\/li>\n<\/ul>\n\n\n\n<div style=\"height:24px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized centered\"><img decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/setting-up-pyquery-1024x576.png\" alt=\"A drawn person works on gears with the title &quot;Setting Up PyQuery&quot;.\" class=\"wp-image-55198\" style=\"object-fit:cover\" srcset=\"https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/setting-up-pyquery-1024x576.png 1024w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/setting-up-pyquery-300x169.png 300w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/setting-up-pyquery-768x432.png 768w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/setting-up-pyquery-1536x864.png 1536w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/setting-up-pyquery-600x338.png 600w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/setting-up-pyquery.png 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<div style=\"height:24px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Setting Up PyQuery<\/h2>\n\n\n\n<p>For starters, you will need to install PyQuery. This can be done easily using pip. Open your terminal or command prompt and run the following command:<\/p>\n\n\n\n<div style=\"height:24px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#e0def4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>pip install pyquery<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">pip install pyquery<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<p>This will download and install PyQuery along with its dependencies, including lxml, which is essential for parsing HTML documents efficiently.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Basic Setup<\/h3>\n\n\n\n<p>With PyQuery installed, you can now start using it in your Python scripts. You will find a basic setup guide to help you get started.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Import PyQuery<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<p>Begin by importing PyQuery within your Python script. This can be done using this command:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#e0def4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>from pyquery import Pyquery as pq<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #3E8FB0\">from<\/span><span style=\"color: #E0DEF4\"> pyquery <\/span><span style=\"color: #3E8FB0\">import<\/span><span style=\"color: #E0DEF4\"> Pyquery <\/span><span style=\"color: #3E8FB0\">as<\/span><span style=\"color: #E0DEF4\"> pq<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"2\" class=\"wp-block-list\">\n<li><strong>Loading HTML content<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<p>PyQuery can load HTML content from a variety of sources including strings and files. If you have HTML content as a string, you can pass it directly to PyQuery using this:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>html = '&lt;html>&lt;body>&lt;h1>Hello, World!&lt;\/h1>&lt;\/body>&lt;\/html>'\nd = pq(html)<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">html <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;&lt;html&gt;&lt;body&gt;&lt;h1&gt;Hello, World!&lt;\/h1&gt;&lt;\/body&gt;&lt;\/html&gt;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">html<\/span><span style=\"color: #908CAA\">)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<p>If your HTML content is in a file however, you can read the file and then pass the content to PyQuery using this:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>with open('example.html', 'r') as f:\n    html = f.read()\nd = pq(html)<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #3E8FB0\">with<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #EB6F92; font-style: italic\">open<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;example.html&#39;<\/span><span style=\"color: #908CAA\">,<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;r&#39;<\/span><span style=\"color: #908CAA\">)<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #3E8FB0\">as<\/span><span style=\"color: #E0DEF4\"> f<\/span><span style=\"color: #908CAA\">:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    html <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> f<\/span><span style=\"color: #908CAA\">.<\/span><span style=\"color: #E0DEF4\">read<\/span><span style=\"color: #908CAA\">()<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">html<\/span><span style=\"color: #908CAA\">)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"3\" class=\"wp-block-list\">\n<li><strong>Loading from a URL<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<p>PyQuery can load HTML content directly from a URL. This can be useful for <a href=\"https:\/\/proxidize.com\/use-cases\/web-scraping\/\">web scraping<\/a> tasks where you need to fetch and parse content from a web page.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#e0def4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>d = pq(url='http:\/\/example.com')<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #C4A7E7; font-style: italic\">url<\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #F6C177\">&#39;http:\/\/example.com&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"4\" class=\"wp-block-list\">\n<li><strong>Basic Operations<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<p>After loading the HTML content, you can start performing basic operations including selecting elements and extracting text. To find elements, use CSS selectors within the HTML document. For example, to select an H1 tag and print its text content, use this code:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#e0def4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>print(d('h1').text())<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;h1&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">())<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<p>To extract and manipulate attributes, you could use a similar looking code. For example, to extract a class attribute, use this:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#e0def4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>print(d('h1').attr('class'))<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;h1&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">attr<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;class&#39;<\/span><span style=\"color: #908CAA\">))<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<p>By following these steps, you will have PyQuery set up and ready to parse and manipulate HTML content. This basic setup provides a foundation for more advanced methods, giving you the ability to take full advantage of PyQuery\u2019s features.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Selecting Multiple Elements<\/h3>\n\n\n\n<p>While the section above covered how to select singular elements, let us explore how you would go about selecting multiple elements at a time to help expand your projects and lessen the need to extract each element one by one. In this section, we will cover how you can select all elements of a type, including selecting elements by class and attribute, as well as how to combine selectors and select nested elements to get the most out of your code.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Selecting All Elements of One Type<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<p>You can select all the elements of a specific type by specifying the tag name. This code will give an example of how to do this by selecting all the &lt;p&gt; tags.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(2 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>from pyquery import PyQuery as pq\n\nhtml = '''\n&lt;html>\n    &lt;body>\n        &lt;p>Paragraph 1&lt;\/p>\n        &lt;p>Paragraph 2&lt;\/p>\n        &lt;p>Paragraph 3&lt;\/p>\n    &lt;\/body>\n&lt;\/html>\n'''\nd = pq(html)\nparagraphs = d('p')\nfor p in paragraphs:\n    print(pq(p).text())\n# Outputs:\n# Paragraph 1\n# Paragraph 2\n# Paragraph 3<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #3E8FB0\">from<\/span><span style=\"color: #E0DEF4\"> pyquery <\/span><span style=\"color: #3E8FB0\">import<\/span><span style=\"color: #E0DEF4\"> PyQuery <\/span><span style=\"color: #3E8FB0\">as<\/span><span style=\"color: #E0DEF4\"> pq<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">html <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;p&gt;Paragraph 1&lt;\/p&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;p&gt;Paragraph 2&lt;\/p&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;p&gt;Paragraph 3&lt;\/p&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;\/body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;\/html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">html<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">paragraphs <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;p&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">for<\/span><span style=\"color: #E0DEF4\"> p <\/span><span style=\"color: #3E8FB0\">in<\/span><span style=\"color: #E0DEF4\"> paragraphs<\/span><span style=\"color: #908CAA\">:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">p<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">())<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Paragraph 1<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Paragraph 2<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Paragraph 3<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"2\" class=\"wp-block-list\">\n<li><strong>Selecting Elements by Class<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<p>You can select elements that share common class attributes. This can be done as such:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(2 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>html = '''\n&lt;html>\n    &lt;body>\n        &lt;div class=\"content\">Content 1&lt;\/div>\n        &lt;div class=\"content\">Content 2&lt;\/div>\n        &lt;div class=\"content\">Content 3&lt;\/div>\n    &lt;\/body>\n&lt;\/html>\n'''\nd = pq(html)\ncontents = d('.content')\nfor content in contents:\n    print(pq(content).text())\n# Outputs:\n# Content 1\n# Content 2\n# Content 3 <\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">html <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div class=&quot;content&quot;&gt;Content 1&lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div class=&quot;content&quot;&gt;Content 2&lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div class=&quot;content&quot;&gt;Content 3&lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;\/body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;\/html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">html<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">contents <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;.content&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">for<\/span><span style=\"color: #E0DEF4\"> content <\/span><span style=\"color: #3E8FB0\">in<\/span><span style=\"color: #E0DEF4\"> contents<\/span><span style=\"color: #908CAA\">:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">content<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">())<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Content 1<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Content 2<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Content 3 <\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"3\" class=\"wp-block-list\">\n<li><strong>Selecting Elements by Attribute<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<p>You could select elements based on attribute values. For instance, selecting all &lt;a&gt; tags with a specific href attribute. This is done as such:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(2 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>html = '''\n&lt;html>\n    &lt;body>\n        &lt;a href=\"link1.html\">Link 1&lt;\/a>\n        &lt;a href=\"link2.html\">Link 2&lt;\/a>\n        &lt;a href=\"link3.html\">Link 3&lt;\/a>\n    &lt;\/body>\n&lt;\/html>\n'''\nd = pq(html)\nlinks = d('a&#91;href&#93;')\nfor link in links:\n    print(pq(link).attr('href'))\n# Outputs:\n# link1.html\n# link2.html\n# link3.html<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">html <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;a href=&quot;link1.html&quot;&gt;Link 1&lt;\/a&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;a href=&quot;link2.html&quot;&gt;Link 2&lt;\/a&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;a href=&quot;link3.html&quot;&gt;Link 3&lt;\/a&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;\/body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;\/html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">html<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">links <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;a&#91;href&#93;&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">for<\/span><span style=\"color: #E0DEF4\"> link <\/span><span style=\"color: #3E8FB0\">in<\/span><span style=\"color: #E0DEF4\"> links<\/span><span style=\"color: #908CAA\">:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">link<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">attr<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;href&#39;<\/span><span style=\"color: #908CAA\">))<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> link1.html<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> link2.html<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> link3.html<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"4\" class=\"wp-block-list\">\n<li><strong>Combining Selectors<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<p>You could combine multiple selectors which help narrow down your selection. As an example, selecting all div elements with a specific class and attribute would look something like this:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(2 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>html = '''\n&lt;html>\n    &lt;body>\n        &lt;div class=\"content\" data-id=\"1\">Content 1&lt;\/div>\n        &lt;div class=\"content\" data-id=\"2\">Content 2&lt;\/div>\n        &lt;div class=\"content\" data-id=\"3\">Content 3&lt;\/div>\n    &lt;\/body>\n&lt;\/html>\n'''\nd = pq(html)\nspecific_contents = d('div.content&#91;data-id&#93;')\nfor content in specific_contents:\n    print(pq(content).text())\n# Outputs:\n# Content 1\n# Content 2\n# Content 3  <\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">html <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div class=&quot;content&quot; data-id=&quot;1&quot;&gt;Content 1&lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div class=&quot;content&quot; data-id=&quot;2&quot;&gt;Content 2&lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div class=&quot;content&quot; data-id=&quot;3&quot;&gt;Content 3&lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;\/body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;\/html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">html<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">specific_contents <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div.content&#91;data-id&#93;&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">for<\/span><span style=\"color: #E0DEF4\"> content <\/span><span style=\"color: #3E8FB0\">in<\/span><span style=\"color: #E0DEF4\"> specific_contents<\/span><span style=\"color: #908CAA\">:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">content<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">())<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Content 1<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Content 2<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Content 3  <\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"5\" class=\"wp-block-list\">\n<li><strong>Selecting Nested Elements<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<p>PyQuery allows you to select nested elements by chaining selectors together. For a code example, we will be selecting all span elements within div elements.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(2 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>html = '''\n&lt;html>\n    &lt;body>\n        &lt;div>\n            &lt;span>Span 1&lt;\/span>\n        &lt;\/div>\n        &lt;div>\n            &lt;span>Span 2&lt;\/span>\n        &lt;\/div>\n        &lt;div>\n            &lt;span>Span 3&lt;\/span>\n        &lt;\/div>\n    &lt;\/body>\n&lt;\/html>\n'''\nd = pq(html)\nspans = d('div span')\nfor span in spans:\n    print(pq(span).text())\n# Outputs:\n# Span 1\n# Span 2\n# Span 3<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">html <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">            &lt;span&gt;Span 1&lt;\/span&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">            &lt;span&gt;Span 2&lt;\/span&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">            &lt;span&gt;Span 3&lt;\/span&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;\/body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;\/html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">html<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">spans <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div span&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">for<\/span><span style=\"color: #E0DEF4\"> span <\/span><span style=\"color: #3E8FB0\">in<\/span><span style=\"color: #E0DEF4\"> spans<\/span><span style=\"color: #908CAA\">:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">span<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">())<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Span 1<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Span 2<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Span 3<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<p>By using all these techniques, you will be able to effectively select and work with multiple elements to parse HTML with Python. These methods provide both flexibility and precision in navigating and manipulating document structure.&nbsp;<\/p>\n\n\n\n<div style=\"height:24px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized centered\"><img decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/advanced-pyquery-techniques-1024x576.png\" alt=\"An inverted triangle on a black background that reads &quot;Advanced PyQuery Techniques&quot;.\" class=\"wp-image-55200\" style=\"object-fit:cover\" srcset=\"https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/advanced-pyquery-techniques-1024x576.png 1024w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/advanced-pyquery-techniques-300x169.png 300w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/advanced-pyquery-techniques-768x432.png 768w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/advanced-pyquery-techniques-1536x864.png 1536w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/advanced-pyquery-techniques-600x338.png 600w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/advanced-pyquery-techniques.png 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<div style=\"height:24px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Advanced PyQuery Techniques<\/h2>\n\n\n\n<p>With the basic out of the way, let us explore some more advanced techniques such as traversing and modifying the DOM as well as how to handle multiple elements.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Traversing the DOM<\/h3>\n\n\n\n<p>Traversing the DOM refers to navigating through the HTML document structure. This involves moving from one element to another. PyQuery offers many methods for DOM traversal that allow you to access parent, child, and sibling elements easily.&nbsp;<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Parent Elements:<\/strong> use the `.parent()` method<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(2 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>from pyquery import PyQuery as pq\n\nhtml = '''\n&lt;html>\n    &lt;body>\n        &lt;div>\n            &lt;p>Paragraph inside div&lt;\/p>\n        &lt;\/div>\n    &lt;\/body>\n&lt;\/html>\n'''\nd = pq(html)\nparent = d('p').parent()\nprint(parent)  # Outputs: &lt;div>...&lt;\/div><\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #3E8FB0\">from<\/span><span style=\"color: #E0DEF4\"> pyquery <\/span><span style=\"color: #3E8FB0\">import<\/span><span style=\"color: #E0DEF4\"> PyQuery <\/span><span style=\"color: #3E8FB0\">as<\/span><span style=\"color: #E0DEF4\"> pq<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">html <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">            &lt;p&gt;Paragraph inside div&lt;\/p&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;\/body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;\/html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">html<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">parent <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;p&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">parent<\/span><span style=\"color: #908CAA\">()<\/span><\/span>\n<span class=\"line\"><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">parent<\/span><span style=\"color: #908CAA\">)<\/span><span style=\"color: #E0DEF4\">  <\/span><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs: &lt;div&gt;...&lt;\/div&gt;<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"2\" class=\"wp-block-list\">\n<li><strong>Child Element:<\/strong> use the `.children()` method<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>children = d('div').children()\nfor child in children:\n    print(pq(child).text())\n# Outputs:\n# Paragraph inside div<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">children <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">children<\/span><span style=\"color: #908CAA\">()<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">for<\/span><span style=\"color: #E0DEF4\"> child <\/span><span style=\"color: #3E8FB0\">in<\/span><span style=\"color: #E0DEF4\"> children<\/span><span style=\"color: #908CAA\">:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">child<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">())<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Paragraph inside div<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"3\" class=\"wp-block-list\">\n<li><strong>Sibling Element:<\/strong> use the `.siblings()` method<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(2 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>html = '''\n&lt;html>\n    &lt;body>\n        &lt;div>First div&lt;\/div>\n        &lt;div>Second div&lt;\/div>\n        &lt;div>Third div&lt;\/div>\n    &lt;\/body>\n&lt;\/html>\n'''\nd = pq(html)\nfirst_div = d('div').eq(0)\nsiblings = first_div.siblings()\nfor sibling in siblings:\n    print(pq(sibling).text())\n# Outputs:\n# Second div\n# Third div<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">html <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div&gt;First div&lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div&gt;Second div&lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">        &lt;div&gt;Third div&lt;\/div&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">    &lt;\/body&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&lt;\/html&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6C177\">&#39;&#39;&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">html<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">first_div <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">eq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #EA9A97\">0<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">siblings <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> first_div<\/span><span style=\"color: #908CAA\">.<\/span><span style=\"color: #E0DEF4\">siblings<\/span><span style=\"color: #908CAA\">()<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">for<\/span><span style=\"color: #E0DEF4\"> sibling <\/span><span style=\"color: #3E8FB0\">in<\/span><span style=\"color: #E0DEF4\"> siblings<\/span><span style=\"color: #908CAA\">:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">sibling<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">())<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Second div<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Third div<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"4\" class=\"wp-block-list\">\n<li><strong>Next and Previous Elements:<\/strong> to select the next and previous sibling element use the `.next()` and `.prev()` methods<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>next_element = d('div:first').next()\nprint(next_element.text())  # Outputs: Second div\n\nprevious_element = d('div:last').prev()\nprint(previous_element.text())  # Outputs: Second div<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">next_element <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div:first&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">next<\/span><span style=\"color: #908CAA\">()<\/span><\/span>\n<span class=\"line\"><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">next_element<\/span><span style=\"color: #908CAA\">.<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">())<\/span><span style=\"color: #E0DEF4\">  <\/span><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs: Second div<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">previous_element <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div:last&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">prev<\/span><span style=\"color: #908CAA\">()<\/span><\/span>\n<span class=\"line\"><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">previous_element<\/span><span style=\"color: #908CAA\">.<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">())<\/span><span style=\"color: #E0DEF4\">  <\/span><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs: Second div<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<div style=\"height:34px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\">Modifying the DOM<\/h3>\n\n\n\n<p>PyQuery gives you the ability to modify the DOM by adding, removing, or changing elements and their attributes as well as changing text and HTML content.&nbsp;<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Adding Elements:<\/strong> use the `.append()`, `.prepend()`, `.after()`, or `.before()` methods<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>d('body').append('&lt;p>New paragraph at the end&lt;\/p>')\nd('body').prepend('&lt;p>New paragraph at the beginning&lt;\/p>')\nd('div:first').after('&lt;p>New paragraph after first div&lt;\/p>')\nd('div:last').before('&lt;p>New paragraph before last div&lt;\/p>')<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;body&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">append<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;&lt;p&gt;New paragraph at the end&lt;\/p&gt;&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;body&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">prepend<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;&lt;p&gt;New paragraph at the beginning&lt;\/p&gt;&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div:first&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">after<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;&lt;p&gt;New paragraph after first div&lt;\/p&gt;&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div:last&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">before<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;&lt;p&gt;New paragraph before last div&lt;\/p&gt;&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"2\" class=\"wp-block-list\">\n<li><strong>Removing Elements:<\/strong> use the `.remove()` method<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#e0def4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>d('div').remove()<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">remove<\/span><span style=\"color: #908CAA\">()<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"3\" class=\"wp-block-list\">\n<li><strong>Changing Attributes:<\/strong> use the `.attr()` method<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#e0def4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>d('div:first').attr('class', 'new-class')<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div:first&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">attr<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;class&#39;<\/span><span style=\"color: #908CAA\">,<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;new-class&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"4\" class=\"wp-block-list\">\n<li><strong>Changing Text Content:<\/strong> use the `.text()` method<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#e0def4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>d('div:first').text('Updated text content')<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div:first&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;Updated text content&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"5\" class=\"wp-block-list\">\n<li><strong>Changing HTML Content:<\/strong> use the `.html()` method<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#e0def4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>d('div:first').html('&lt;span>New HTML content&lt;\/span>')<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div:first&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">html<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;&lt;span&gt;New HTML content&lt;\/span&gt;&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<div style=\"height:34px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\">Handling Multiple Elements<\/h3>\n\n\n\n<p>PyQuery is such a powerful tool that you could work with multiple elements at once. You can perform operations on collections of elements efficiently. We previously discussed how you can select multiple elements. This section will cover how you could iterate over elements, apply changes to multiple elements, filter elements, and map over elements.&nbsp;<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Iterating over Elements:<\/strong> use the `.each()` method to iterate over a set<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>d('div').each(lambda i, e: print(pq(e).text()))\n# Outputs:\n# First div\n# Second div\n# Third div<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">each<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #3E8FB0\">lambda<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #C4A7E7; font-style: italic\">i<\/span><span style=\"color: #908CAA\">,<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #C4A7E7; font-style: italic\">e<\/span><span style=\"color: #908CAA\">:<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">e<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">()))<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> First div<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Second div<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Third div<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"2\" class=\"wp-block-list\">\n<li><strong>Applying Changes to Multiple Elements:<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>d('div').addClass('highlight')\n# Adds the 'highlight' class to all &lt;div> elements<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">addClass<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;highlight&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Adds the &#39;highlight&#39; class to all &lt;div&gt; elements<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"3\" class=\"wp-block-list\">\n<li><strong>Filtering Elements:<\/strong> use the `.filter()` method to refine your selection<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>filtered = d('div').filter(lambda i, this: pq(this).text() == 'Second div')\nprint(filtered)  # Outputs: &lt;div>Second div&lt;\/div><\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">filtered <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">filter<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #3E8FB0\">lambda<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #C4A7E7; font-style: italic\">i<\/span><span style=\"color: #908CAA\">,<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #C4A7E7; font-style: italic\">this<\/span><span style=\"color: #908CAA\">:<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">this<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">()<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #3E8FB0\">==<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;Second div&#39;<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">filtered<\/span><span style=\"color: #908CAA\">)<\/span><span style=\"color: #E0DEF4\">  <\/span><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs: &lt;div&gt;Second div&lt;\/div&gt;<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<ol start=\"4\" class=\"wp-block-list\">\n<li><strong>Mapping Over Elements:<\/strong> use the `.map()` method to transform a set of elements:<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(1 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>texts = d('div').map(lambda i, this: pq(this).text().upper())\nfor text in texts:\n    print(text)\n# Outputs:\n# FIRST DIV\n# SECOND DIV\n# THIRD DIV<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #E0DEF4\">texts <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> d<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;div&#39;<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">map<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #3E8FB0\">lambda<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #C4A7E7; font-style: italic\">i<\/span><span style=\"color: #908CAA\">,<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #C4A7E7; font-style: italic\">this<\/span><span style=\"color: #908CAA\">:<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">this<\/span><span style=\"color: #908CAA\">).<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">().<\/span><span style=\"color: #E0DEF4\">upper<\/span><span style=\"color: #908CAA\">())<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">for<\/span><span style=\"color: #E0DEF4\"> text <\/span><span style=\"color: #3E8FB0\">in<\/span><span style=\"color: #E0DEF4\"> texts<\/span><span style=\"color: #908CAA\">:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #EB6F92; font-style: italic\">print<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> Outputs:<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> FIRST DIV<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> SECOND DIV<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA; font-style: italic\">#<\/span><span style=\"color: #6E6A86; font-style: italic\"> THIRD DIV<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<div style=\"height:24px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized centered\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/using-proxies-with-pyquery-1024x576.png\" alt=\"A drawn man types on a laptop on top of a larger laptop with the title &quot;Using Proxies with PyQuery&quot;.\" class=\"wp-image-55199\" style=\"object-fit:cover\" srcset=\"https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/using-proxies-with-pyquery-1024x576.png 1024w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/using-proxies-with-pyquery-300x169.png 300w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/using-proxies-with-pyquery-768x432.png 768w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/using-proxies-with-pyquery-1536x864.png 1536w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/using-proxies-with-pyquery-600x338.png 600w, https:\/\/proxidize.com\/wp-content\/uploads\/2024\/08\/using-proxies-with-pyquery.png 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<div style=\"height:24px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Using Proxies with PyQuery<\/h2>\n\n\n\n<p>An integral part of web scraping and data extraction is to use a proxy. Proxies can help by bypassing geographical restrictions, avoiding IP blocks, and enhancing access to web content. By using PyQuery with a <a href=\"https:\/\/proxidize.com\/proxy-server\/\">proxy server<\/a>, you can gather data from various sources while minimizing the risk of being blocked by websites due to an increase in requests. You have a choice between using a residential proxy, a datacenter, or a <a href=\"https:\/\/proxidize.com\/proxy-server\/mobile-proxy\/\">mobile proxy<\/a>, however, using a mobile proxy will provide the lowest chance of being blocked.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><strong>Setting Up a Proxy:<\/strong> You can configure your HTTP requests to use a proxy by passing proxy settings to the requests library. This can be done by inserting this code block:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(2 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>import requests\nfrom pyquery import PyQuery as pq\n\nproxy = {\n    'http': 'http:\/\/your.proxy.server:port',\n    'https': 'http:\/\/your.proxy.server:port'\n}\n\nresponse = requests.get('http:\/\/example.com', proxies=proxy)\nd = pq(response.text)<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #3E8FB0\">import<\/span><span style=\"color: #E0DEF4\"> requests<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">from<\/span><span style=\"color: #E0DEF4\"> pyquery <\/span><span style=\"color: #3E8FB0\">import<\/span><span style=\"color: #E0DEF4\"> PyQuery <\/span><span style=\"color: #3E8FB0\">as<\/span><span style=\"color: #E0DEF4\"> pq<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">proxy <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #908CAA\">{<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #F6C177\">&#39;http&#39;<\/span><span style=\"color: #908CAA\">:<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;http:\/\/your.proxy.server:port&#39;<\/span><span style=\"color: #908CAA\">,<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #F6C177\">&#39;https&#39;<\/span><span style=\"color: #908CAA\">:<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;http:\/\/your.proxy.server:port&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA\">}<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">response <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> requests<\/span><span style=\"color: #908CAA\">.<\/span><span style=\"color: #E0DEF4\">get<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;http:\/\/example.com&#39;<\/span><span style=\"color: #908CAA\">,<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #C4A7E7; font-style: italic\">proxies<\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\">proxy<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">response<\/span><span style=\"color: #908CAA\">.<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<p><strong>Rotating Proxies:<\/strong> To further avoid being blocked, you can rotate proxies using a list of proxy servers. Most proxy providers offer the ability to create a proxy pool that would rotate your IP, hiding your parsing tasks from detection. You can set up rotating proxies using this code block:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#e0def4;--cbp-line-number-width:calc(2 * 0.6 * .75rem);--cbp-line-highlight-color:rgba(215, 211, 255, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span role=\"button\" tabindex=\"0\" style=\"color:#232136;display:none;background-color:#e0def4\" aria-label=\"Copy\" data-copied-text=\"Copied!\" data-has-text-button=\"textSimple\" data-inside-header-type=\"none\" aria-live=\"polite\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>import random\nimport requests\nfrom pyquery import PyQuery as pq\n\nproxies = &#91;\n    'http:\/\/proxy1.server:port',\n    'http:\/\/proxy2.server:port',\n    'http:\/\/proxy3.server:port'\n&#93;\n\nproxy = {'http': random.choice(proxies), 'https': random.choice(proxies)}\nresponse = requests.get('http:\/\/example.com', proxies=proxy)\nd = pq(response.text)<\/textarea><\/pre><span class=\"cbp-btn-text\">Copy<\/span><\/span><pre class=\"shiki rose-pine-moon\" style=\"background-color: #232136\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #3E8FB0\">import<\/span><span style=\"color: #E0DEF4\"> random<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">import<\/span><span style=\"color: #E0DEF4\"> requests<\/span><\/span>\n<span class=\"line\"><span style=\"color: #3E8FB0\">from<\/span><span style=\"color: #E0DEF4\"> pyquery <\/span><span style=\"color: #3E8FB0\">import<\/span><span style=\"color: #E0DEF4\"> PyQuery <\/span><span style=\"color: #3E8FB0\">as<\/span><span style=\"color: #E0DEF4\"> pq<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">proxies <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #908CAA\">&#91;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #F6C177\">&#39;http:\/\/proxy1.server:port&#39;<\/span><span style=\"color: #908CAA\">,<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #F6C177\">&#39;http:\/\/proxy2.server:port&#39;<\/span><span style=\"color: #908CAA\">,<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">    <\/span><span style=\"color: #F6C177\">&#39;http:\/\/proxy3.server:port&#39;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #908CAA\">&#93;<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">proxy <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #908CAA\">{<\/span><span style=\"color: #F6C177\">&#39;http&#39;<\/span><span style=\"color: #908CAA\">:<\/span><span style=\"color: #E0DEF4\"> random<\/span><span style=\"color: #908CAA\">.<\/span><span style=\"color: #E0DEF4\">choice<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">proxies<\/span><span style=\"color: #908CAA\">),<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #F6C177\">&#39;https&#39;<\/span><span style=\"color: #908CAA\">:<\/span><span style=\"color: #E0DEF4\"> random<\/span><span style=\"color: #908CAA\">.<\/span><span style=\"color: #E0DEF4\">choice<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">proxies<\/span><span style=\"color: #908CAA\">)}<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">response <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> requests<\/span><span style=\"color: #908CAA\">.<\/span><span style=\"color: #E0DEF4\">get<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #F6C177\">&#39;http:\/\/example.com&#39;<\/span><span style=\"color: #908CAA\">,<\/span><span style=\"color: #E0DEF4\"> <\/span><span style=\"color: #C4A7E7; font-style: italic\">proxies<\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\">proxy<\/span><span style=\"color: #908CAA\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #E0DEF4\">d <\/span><span style=\"color: #3E8FB0\">=<\/span><span style=\"color: #E0DEF4\"> pq<\/span><span style=\"color: #908CAA\">(<\/span><span style=\"color: #E0DEF4\">response<\/span><span style=\"color: #908CAA\">.<\/span><span style=\"color: #E0DEF4\">text<\/span><span style=\"color: #908CAA\">)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p>Throughout this guide to PyQuery, we explored parsing HTML with Python, including a simple setup with some code examples to show how this valuable tool can assist and simplify your parsing tasks. We covered some more advanced techniques such as manipulating the DOM and handling multiple elements. We also reminded you how valuable using a proxy is when performing these tasks as the risk of getting detected can cause your IP to be blocked from accessing the site.&nbsp;<\/p>\n\n\n\n<p>After you have parsed your links, if you are interested in scraping the information, we have written a few articles detailing how to start web scraping using BeautifulSoup and a guide on <a href=\"https:\/\/proxidize.com\/use-cases\/web-scraping-with-selenium\/\">web scraping with Selenium<\/a> in Python. With the knowledge you gained from this article on how to parse HTML with Python mixed with the information on BeautifulSoup or Selenium scraping, you will be able to automate your data collection tasks and save countless hours.&nbsp;<\/p>\n","protected":false},"author":2627,"featured_media":76328,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","format":"standard","categories":[266],"tags":[],"class_list":["post-55174","blog","type-blog","status-publish","format-standard","has-post-thumbnail","hentry","category-tech-tutorials-and-programming"],"acf":[],"_links":{"self":[{"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/blog\/55174","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/blog"}],"about":[{"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/types\/blog"}],"author":[{"embeddable":true,"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/users\/2627"}],"replies":[{"embeddable":true,"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/comments?post=55174"}],"version-history":[{"count":3,"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/blog\/55174\/revisions"}],"predecessor-version":[{"id":84847,"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/blog\/55174\/revisions\/84847"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/media\/76328"}],"wp:attachment":[{"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/media?parent=55174"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/categories?post=55174"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/proxidize.com\/wp-json\/wp\/v2\/tags?post=55174"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}