html2text to plain text parsing:

html2text

like parsing tables:

$ html2text foo.txt | column -ts'|'


using xpath (XML::XPath perl module), see example here

xpath

XML::XPath

perl or Python (see @Gilles example)

for parsing multiple files at once, see: How to parse hundred html source code files in shell?

Related (why you shouldn't use regex match):

I agree that the right tool to parse XML and especially HTML is a parser and not a regular expression engine. However, like others have pointed out, sometimes using a regex is quicker, easier, and gets the job done if you know the data format.

Microsoft actually has a section of Best Practices for Regular Expressions in the .NET Framework and specifically talks about Consider[ing] the Input Source.

Regular Expressions do have limitations, but have you considered the following?

The .NET framework is unique when it comes to regular expressions in that it supports Balancing Group Definitions.

For this reason, I believe you CAN parse XML using regular expressions. Note however, that it must be valid XML (browsers are very forgiving of HTML and allow bad XML syntax inside HTML). This is possible since the "Balancing Group Definition" will allow the regular expression engine to act as a PDA.

Quote from article 1 cited above:

.NET Regular Expression Engine

As described above properly balanced constructs cannot be described by
a regular expression. However, the .NET regular expression engine
provides a few constructs that allow balanced constructs to be
recognized.

(?)

(?)

(?(group)yes|no)

These constructs allow for a .NET regular expression to emulate a
restricted PDA by essentially allowing simple versions of the stack
operations: push, pop and empty. The simple operations are pretty much
equivalent to increment, decrement and compare to zero respectively.
This allows for the .NET regular expression engine to recognize a
subset of the context-free languages, in particular the ones that only
require a simple counter. This in turn allows for the non-traditional
.NET regular expressions to recognize individual properly balanced
constructs.

Consider the following regular expression:

(?=)
(?>
|
]*/> |
(?]*[^/]>) |
(?]*[^/]>) |
[^]*
)*
(?(opentag)(?!))


Use the flags:

(?=) # match start with

# atomic group / don't backtrack (faster)
| # match xml / html comment
]*/> | # self closing tag
(?]*[^/]>) | # push opening xml tag
(?]*[^/]>) | # pop closing xml tag
[^]* # something between tags
)* # match as many xml tags as possible
(?(opentag)(?!)) # ensure no 'opentag' groups are on stack






You can try this at A Better .NET Regular Expression Tester.






I used the sample source of:













• stuff...


• more stuff


• 
  
  
  still more
  

  
  
  • Another >ul<, oh my!
  
  
  • ...
  
  

  
  
  
  </li>
  </ul>
  </div>
  </body>
  </html>
  
  
  
  
  

  This found the match:

  
  

  <ul id="matchMe" type="square">
<li>stuff...</li>
<li>more stuff</li>
<li>


  
  still more
  

  
  
  • Another >ul<, oh my!
  
  
  • ...
  
  

  
  
  
</li>
</ul>

  
  
  
  

  although it actually came out like this:

  
  

  <ul id="matchMe" type="square"> <li>stuff...</li> <li>more stuff</li> <li>

  still more

  • Another >ul<, oh my!
  • ...
  </li> </ul>

  
  
  
  

  Lastly, I really enjoyed Jeff Atwood's article: Parsing Html The Cthulhu Way. Funny enough, it cites the answer to this question that currently has over 4k votes.

  
  
  
  
  
  

  
  System.Text is not part of C#. It's part of .NET.
  – John Saunders
  Feb 2 '12 at 19:07
  

  
  
  
  

  System.Text

  
  
  
  
  
  

  
  In the first line of your regex ((?=<uls*id="matchMe"s*type="square"s*>) # match start with <ul id="matchMe"...), in between "<ul" and "id" should be s+, not s*, unless you want it to match <ulid=... ;)
  – C0deH4cker
  Jul 6 '12 at 2:49
  

  
  
  
  

  (?=<uls*id="matchMe"s*type="square"s*>) # match start with <ul id="matchMe"...

  
  

  s+

  
  

  s*

  
  
  
  
  
  

  
  @C0deH4cker You are correct, the expression should have s+ instead of s*.
  – Sam
  Jul 6 '12 at 22:33
  

  
  
  
  

  s+

  
  

  s*

  
  
  
  
  
  

  
  Not that I really understand it, but I think your regex fails on <img src="images/pic.jpg" />
  – Scheintod
  Sep 27 '13 at 17:05
  

  
  
  
  

  <img src="images/pic.jpg" />

  
  
  
  
  
  

  
  @Scheintod Thank you for the comment. I updated the code. The previous expression failed for self closing tags that had a / somewhere inside which failed for your <img src="images/pic.jpg" /> html.
  – Sam
  Sep 27 '13 at 19:00
  

  
  
  
  

  /

  
  

  <img src="images/pic.jpg" />

  
  
  
  

  I suggest using QueryPath for parsing XML and HTML in PHP. It's basically much the same syntax as jQuery, only it's on the server side.

  
  
  
  
  
  

  
  @Kyle—jQuery does not parse XML, it uses the client's built–in parser (if there is one). Therefore you do not need jQuery to do it, but as little as two lines of plain old JavaScript. If there is no built–in parser, jQuery will not help.
  – RobG
  Oct 31 '13 at 6:25
  
  

  
  
  
  
  
  
  
  

  
  @RobG Actually jQuery uses the DOM, not the built-in parser.
  – Qix
  Sep 22 '14 at 3:49
  

  
  
  
  
  
  
  
  

  
  @Qix—you'd better tell the authors of the documentation then: "jQuery.parseXML uses the native parsing function of the browser…". Source: jQuery.parseXML()
  – RobG
  Sep 22 '14 at 5:01
  
  

  
  
  
  
  
  
  
  

  
  Having come here from the meme question (meta.stackexchange.com/questions/19478/the-many-memes-of-meta/…), I love that one of the answers is 'Use jQuery'
  – Jorn
  Apr 1 '16 at 21:09
  

  
  
  
  
  
  

  While the answers that you can't parse HTML with regexes are correct, they don't apply here. The OP just wants to parse one HTML tag with regexes, and that is something that can be done with a regular expression.

  
  
  
  

  The suggested regex is wrong, though:

  
  

  <([a-z]+) *[^/]*?>


  
  
  
  

  If you add something to the regex, by backtracking it can be forced to match silly things like <a >>, [^/] is too permissive. Also note that <space>*[^/]* is redundant, because the [^/]* can also match spaces.

  
  

  <a >>

  
  

  [^/]

  
  

  <space>*[^/]*

  
  

  [^/]*

  
  
  
  

  My suggestion would be

  
  

  <([a-z]+)[^>]*(?<!/)>


  
  
  
  

  Where (?<! ... ) is (in Perl regexes) the negative look-behind. It reads "a <, then a word, then anything that's not a >, the last of which may not be a /, followed by >".

  
  

  (?<! ... )

  
  
  
  

  Note that this allows things like <a/ > (just like the original regex), so if you want something more restrictive, you need to build a regex to match attribute pairs separated by spaces.

  
  

  <a/ >

  
  
  
  
  
  

  
  +1 for noting that the question is not about parsing full (X)HTML, it's about matching (X)HTML open tags.
  – LarsH
  Sep 8 '12 at 2:26
  

  
  
  
  
  
  
  
  

  
  Something else most of the answers seem to ignore, is that an HTML parser can very well use regular expressions in its implementation for parts of HTML, and I would be surprised if most parsers didn't do this.
  – Thayne
  Mar 26 '15 at 19:15
  

  
  
  
  
  
  
  
  

  
  @Thayne Exactly. When parsing individual tags, a regular expression is the right tool for the job. It is quite ridiculous that one has to scroll halfway down the page to find a reasonable answer. The accepted answer is incorrect because it mixes up lexing and parsing.
  – kasperd
  Nov 22 '15 at 10:26
  

  
  
  
  
  
  
  
  

  
  The answer given here will fail when an attribute value contains a '>' or '/' character.
  – Martin L
  Apr 21 '16 at 8:14
  

  
  
  
  
  
  
  
  

  
  This will work incorrectly on HTML containing comments or CData sections. It will also not work correctly if a quoted attribute contains a > character. I agree what OP suggest can be done with a regex, but the one presented here is far to simplistic.
  – JacquesB
  Jul 30 '17 at 10:14
  

  
  
  
  

  >

  
  
  
  

  Try:

  
  

  <([^s]+)(s[^>]*?)?(?<!/)>


  
  
  
  

  It is similar to yours, but the last > must not be after a slash, and also accepts h1.

  
  

  >

  
  

  h1

  
  
  
  
  
  

  
  <a href="foo" title="5>3"> Oops </a>
  – Gareth
  Nov 13 '09 at 23:11
  

  
  
  
  
  
  
  
  

  
  That is very true, and I did think about it, but I assumed the > symbol is properly escaped to &gt;.
  – Kobi
  Nov 13 '09 at 23:16
  

  
  
  
  

  >

  
  
  
  
  
  

  
  > is valid in an attribute value. Indeed, in the ‘canonical XML’ serialisation you must not use >. (Which isn't entirely relevant, except to emphasise that > in an attribute value is not at all an unusual thing.)
  – bobince
  Nov 14 '09 at 0:15
  

  
  
  
  

  >

  
  

  >

  
  

  >

  
  
  
  
  
  

  
  @Kobi: what does the exlamation mark (the one you placed tpward the end) mean in a regexp?
  – Marco Demaio
  Apr 30 '11 at 17:16
  
  

  
  
  
  
  
  
  
  

  
  @bobince: are u sure? I don't understand anymore, so is this valid HTML too:

  ">hello</div>
  – Marco Demaio
  Apr 30 '11 at 17:31
  
  

  
  
  
  
  ">hello</div>
  
  
  
  

  Sun Tzu, an ancient Chinese strategist, general, and philosopher, said:

  
  
  
  

  It is said that if you know your enemies and know yourself, you can win a hundred battles without a single loss.
  If you only know yourself, but not your opponent, you may win or may lose.
  If you know neither yourself nor your enemy, you will always endanger yourself.

  
  
  
  

  In this case your enemy is HTML and you are either yourself or regex. You might even be Perl with irregular regex. Know HTML. Know yourself.

  
  
  
  

  I have composed a haiku describing the nature of HTML.

  
  

  HTML has
complexity exceeding
regular language.


  
  
  
  

  I have also composed a haiku describing the nature of regex in Perl.

  
  

  The regex you seek
is defined within the phrase
<([a-zA-Z]+)(?:[^>]*[^/]*)?>


  
  

  <?php
$selfClosing = explode(',', 'area,base,basefont,br,col,frame,hr,img,input,isindex,link,meta,param,embed');

$html = '
<p><a href="#">foo</a></p>
<hr/>
<br/>


  name
  ';

$dom = new DOMDocument();
$dom->loadHTML($html);
$els = $dom->getElementsByTagName('*');
foreach ( $els as $el ) {
$nodeName = strtolower($el->nodeName);
if ( !in_array( $nodeName, $selfClosing ) ) {
var_dump( $nodeName );
}
}

  
  
  
  

  Output:

  
  

  string(4) "html"
string(4) "body"
string(1) "p"
string(1) "a"
string(3) "div"


  
  
  
  

  Basically just define the element node names that are self closing, load the whole html string into a DOM library, grab all elements, loop through and filter out ones which aren't self closing and operate on them.

  
  
  
  

  I'm sure you already know by now that you shouldn't use regex for this purpose.

  
  
  
  
  
  

  
  If you're dealing with real XHTML then append getElementsByTagName with NS and specify the namespace.
  – meder omuraliev
  Nov 15 '09 at 14:39
  

  
  
  
  

  NS

  
  
  
  

  I don't know your exact need for this, but if you are also using .NET, couldn't you use Html Agility Pack?

  
  
  
  

  Excerpt:

  
  
  
  

  It is a .NET code library that allows
  you to parse "out of the web" HTML
  files. The parser is very tolerant
  with "real world" malformed HTML.

  
  
  
  

  You want the first > not preceded by a /. Look here for details on how to do that. It's referred to as negative lookbehind.

  
  

  >

  
  

  /

  
  
  
  

  However, a naïve implementation of that will end up matching <bar/></foo> in this example document

  
  

  <bar/></foo>

  
  

  <foo><bar/></foo>


  
  
  
  

  Can you provide a little more information on the problem you're trying to solve? Are you iterating through tags programatically?

  
  
  
  
  
  

  
  Yep, I sure am. Determining all the tags that are currently open, then compare that against the closed tags in a separate array. RegEx hurts my brain.
  – Jeff
  Nov 13 '09 at 23:04
  

  
  
  
  
  
  

  The W3C explains parsing in a pseudo regexp form:
  W3C Link

  
  
  
  

  Follow the var links for QName, S, and Attribute to get a clearer picture.
  Based on that you can create a pretty good regexp to handle things like stripping tags.

  
  

  QName

  
  

  S

  
  

  Attribute

  
  
  
  
  
  

  
  That's not a psuedo regexp form, that's an EBNF form, as specified here: XML spec, appendix 6
  – Rob G
  Feb 11 '15 at 10:34
  
  

  
  
  
  
  
  

  If you need this for PHP:

  
  
  
  

  The PHP DOM functions won't work properly unless it is properly formatted XML. No matter how much better their use is for the rest of mankind.

  
  
  
  

  simplehtmldom is good, but I found it a bit buggy, and it is is quite memory heavy [Will crash on large pages.]

  
  
  
  

  I have never used querypath, so can't comment on its usefulness.

  
  
  
  

  Another one to try is my DOMParser which is very light on resources and I've been using happily for a while. Simple to learn & powerful.

  
  
  
  

  For Python and Java, similar links were posted.

  
  
  
  

  For the downvoters - I only wrote my class when the XML parsers proved unable to withstand real use. Religious downvoting just prevents useful answers from being posted - keep things within perspective of the question, please.

  
  
  
  

  I used a open source tool called HTMLParser before. It's designed to parse HTML in various ways and serves the purpose quite well. It can parse HTML as different treenode and you can easily use its API to get attributes out of the node. Check it out and see if this can help you.

  
  
  
  

  Whenever I need to quickly extract something from an HTML document, I use Tidy to convert it to XML and then use XPath or XSLT to get what I need.
  In your case, something like this:

  
  

  //p/a[@href='foo']


  
  
  
  

  I like to parse HTML with regular expressions. I don't attempt to parse idiot HTML that is deliberately broken. This code is my main parser (Perl edition):

  
  

  $_ = join "",<STDIN>; tr/nr t/ /s; s/</n</g; s/>/>n/g; s/n ?n/n/g;
s/^ ?n//s; s/ $//s; print


  
  
  
  

  It's called htmlsplit, splits the HTML into lines, with one tag or chunk of text on each line. The lines can then be processed further with other text tools and scripts, such as grep, sed, Perl, etc. I'm not even joking :) Enjoy.

  
  
  
  

  It is simple enough to rejig my slurp-everything-first Perl script into a nice streaming thing, if you wish to process enormous web pages. But it's not really necessary.

  
  
  
  

  I bet I will get downvoted for this.

  
  
  
  

  HTML Split

  
  
  
  

  Against my expectation this got some upvotes, so I'll suggest some better regular expressions:

  
  

  /(<.*?>|[^<]+)s*/g # get tags and text
/(w+)="(.*?)"/g # get attibutes


  
  
  
  

  They are good for XML / XHTML.

  
  
  
  

  With minor variations, it can cope with messy HTML... or convert the HTML -> XHTML first.

  
  
  
  

  The best way to write regular expressions is in the Lex / Yacc style, not as opaque one-liners or commented multi-line monstrosities. I didn't do that here, yet; these ones barely need it.

  
  
  
  
  
  

  
  "I don't attempt to parse idiot HTML that is deliberately broken." How does your code know the difference?
  – Kevin Panko
  Jul 26 '11 at 20:38
  

  
  
  
  
  
  
  
  

  
  Well it doesn't matter much if the HTML is broken or not. The thing will still split HTML into tags and text. The only thing that could foul it up is if people include unescaped < or > characters in text or attributes. In practise, my tiny HTML splitter works well. I don't need an enormous monstrosity chock full of heuristics. Simple solutions are not for everyone...!
  – Sam Watkins
  Mar 8 '12 at 3:22
  
  

  
  
  
  
  
  
  
  

  
  I added some simpler regexps for extracting tags, text, and attributes, for XML / XHTML.
  – Sam Watkins
  May 22 '12 at 8:00
  

  
  
  
  
  
  
  
  

  
  (get attributes bug 1) /(w+)="(.*?)"/ assumes double quotes. It will miss values in single quotes. In html version 4 and earlier unquoted value is allowed, if it is a simple word.
  – David Andersson
  Sep 11 '16 at 8:23
  

  
  
  
  

  /(w+)="(.*?)"/

  
  
  
  
  
  

  
  (get attributes bug 2) /(w+)="(.*?)"/ may falsely match text that looks like an attribute within an attribute, e.g. <img title="Nope down='up' for aussies" src="..." />. If applied globally, it will also match such things in ordinary text or in html comments.
  – David Andersson
  Sep 11 '16 at 8:28
  

  
  
  
  

  /(w+)="(.*?)"/

  
  

  <img title="Nope down='up' for aussies" src="..." />

  
  
  
  

  Here's the solution:

  
  

  <?php
// here's the pattern:
$pattern = '/<(w+)(s+(w+)s*=s*('|")(.*?)4s*)*s*(/>|>)/';

// a string to parse:
$string = 'Hello, try clicking <a href="#paragraph">here</a>
<br/>and check out.<hr />
<h2>title</h2>
<a name ="paragraph" rel= "I'm an anchor"></a>
Fine, <span title='highlight the "punch"'>thanks<span>.


  
<br>';

// let's get the occurrences:
preg_match_all($pattern, $string, $matches, PREG_PATTERN_ORDER);

// print the result:
print_r($matches[0]);
?>

  
  
  
  

  To test it deeply, I entered in the string auto-closing tags like:

  
  
  
  

  I also entered tags with:

  
  
  
  

  Should you find something which does not work in the proof of concept above, I am available in analyzing the code to improve my skills.

  
  
  
  

  <EDIT>
  I forgot that the question from the user was to avoid the parsing of self-closing tags.
  In this case the pattern is simpler, turning into this:

  
  

  $pattern = '/<(w+)(s+(w+)s*=s*('|")(.*?)4s*)*s*>/';


  
  
  
  

  The user @ridgerunner noticed that the pattern does not allow unquoted attributes or attributes with no value. In this case a fine tuning brings us the following pattern:

  
  

  $pattern = '/<(w+)(s+(w+)(s*=s*('|"|)(.*?)5s*)?)*s*>/';


  
  
  
  

  </EDIT>

  
  
  
  

  Understanding the pattern

  
  
  
  

  If someone is interested in learning more about the pattern, I provide some line:

  
  
  
  

  Small tip: to better analyze this code it is necessary looking at the source code generated since I did not provide any HTML special characters escaping.

  
  
  
  
  
  

  
  Does not match valid tags having attributes with no value, i.e. <option selected>. Also does not match valid tags with unquoted attribute values, i.e. <p id=10>.
  – ridgerunner
  Jul 25 '11 at 15:01
  
  

  
  
  
  

  <option selected>

  
  

  <p id=10>

  
  
  
  
  
  

  
  @ridgerunner: Thanks very much for your comment. In that case the pattern must change a bit: $pattern = '/<(w+)(s+(w+)(s*=s*('|"|)(.*?)5s*)?)*s*>/'; I tested it and works in case of non-quoted attributes or attributes with no value.
  – Emanuele Del Grande
  Jul 25 '11 at 16:41
  
  

  
  
  
  
  
  
  
  

  
  How about a space before the tag name: < a href="http://wtf.org" > I'm pretty sure it is legal, but you don't match it.
  – Floris
  Oct 5 '13 at 4:58
  

  
  
  
  

  < a href="http://wtf.org" >

  
  
  
  
  
  

  
  NO sorry, whitespaces before a tagname are illegal. Beyond being "pretty sure" why don't you provide some evidences of your objection? Here are mine, w3.org/TR/xml11/#sec-starttags referred to XML 1.1, and you can find the same for HTML 4, 5 and XHTML, as a W3C validation would also warn if you make a test. As a lot of other blah-blah-poets around here, I did not still receive any intelligent argumentation, apart some hundred of minus to my answers, to demonstrate where my code fails according to the rules of contract specified in the question. I would only welcome them.
  – Emanuele Del Grande
  Oct 6 '13 at 18:03
  

  
  
  
  
  
  
  
  

  
  @ridgerunner of course your comment was intelligent and welcome.
  – Emanuele Del Grande
  Oct 6 '13 at 18:09
  

  
  
  
  
  
  

  Here is a PHP based parser that parses HTML using some ungodly regex. As the author of this project, I can tell you it is possible to parse HTML with regex, but not efficient. If you need a server-side solution (as I did for my wp-Typography WordPress plugin), this works.

  
  
  
  
  
  

  
  htmlawed is another PHP project that parses HTML to filter, convert, etc. Has some nice code if you can figure it out!
  – user594694
  May 12 '11 at 19:22
  

  
  
  
  
  
  
  
  

  
  No you can’t parse HTML with regex. But for some subsets, it may work.
  – mirabilos
  Dec 5 '14 at 17:07
  

  
  
  
  
  
  

  There are some nice regexes for replacing HTML with BBCode here. For all you nay-sayers, note that he's not trying to fully parse HTML, just to sanitize it. He can probably afford to kill off tags that his simple "parser" can't understand.

  
  
  
  

  For example:

  
  

  $store =~ s/http:/http:///gi;
$store =~ s/https:/https:///gi;
$baseurl = $store;

if (!$query->param("ascii")) {
$html =~ s/ss+/n/gi;
$html =~ s/<pre(.*?)>(.*?)</pre>/[code]$2[/code]/sgmi;
}

$html =~ s/n//gi;
$html =~ s/rr//gi;
$html =~ s/$baseurl//gi;
$html =~ s/<h[1-7](.*?)>(.*?)</h[1-7]>/n[b]$2[/b]n/sgmi;
$html =~ s/<p>/nn/gi;
$html =~ s/<br(.*?)>/n/gi;
$html =~ s/<textarea(.*?)>(.*?)</textarea>/[code]$2[/code]/sgmi;
$html =~ s/<b>(.*?)</b>/[b]$1[/b]/gi;
$html =~ s/<i>(.*?)</i>/[i]$1[/i]/gi;
$html =~ s/<u>(.*?)</u>/[u]$1[/u]/gi;
$html =~ s/<em>(.*?)</em>/[i]$1[/i]/gi;
$html =~ s/<strong>(.*?)</strong>/[b]$1[/b]/gi;
$html =~ s/<cite>(.*?)</cite>/[i]$1[/i]/gi;
$html =~ s/<font color="(.*?)">(.*?)</font>/[color=$1]$2[/color]/sgmi;
$html =~ s/<font color=(.*?)>(.*?)</font>/[color=$1]$2[/color]/sgmi;
$html =~ s/<link(.*?)>//gi;
$html =~ s/<li(.*?)>(.*?)</li>/[*]$2/gi;
$html =~ s/<ul(.*?)>/[list]/gi;
$html =~ s/</ul>/[/list]/gi;
$html =~ s/

  /n/gi;
$html =~ s/
  /n/gi;
$html =~ s/<td(.*?)>/ /gi;
$html =~ s/<tr(.*?)>/n/gi;

$html =~ s/<img(.*?)src="(.*?)"(.*?)>/[img]$baseurl/$2[/img]/gi;
$html =~ s/<a(.*?)href="(.*?)"(.*?)>(.*?)</a>/[url=$baseurl/$2]$4[/url]/gi;
$html =~ s/[url=$baseurl/http://(.*?)](.*?)[/url]/[url=http://$1]$2[/url]/gi;
$html =~ s/[img]$baseurl/http://(.*?)[/img]/[img]http://$1[/img]/gi;

$html =~ s/<head>(.*?)</head>//sgmi;
$html =~ s/(.*?)//sgmi;
$html =~ s/(.*?)//sgmi;
$html =~ s/<style(.*?)>(.*?)</style>//sgmi;
$html =~ s/<title>(.*?)</title>//sgmi;
$html =~ s/<!--(.*?)-->/n/sgmi;

$html =~ s//////gi;
$html =~ s/http://http:///gi;
$html =~ s/https://https:///gi;

$html =~ s/<(?:[^>'"]*|(['"]).*?1)*>//gsi;
$html =~ s/rr//gi;
$html =~ s/[img]//[img]/gi;
$html =~ s/[url=//[url=/gi;

  
  
  
  
  
  

  
  Don't do this. Please.
  – maletor
  Sep 3 '15 at 2:12
  

  
  
  
  
  
  

  About the question of the RegExp methods to parse (x)HTML, the answer to all of the ones who spoke about some limits is: you have not been trained enough to rule the force of this powerful weapon, since NOBODY here spoke about recursion.

  
  
  
  

  A RegExp-agnostic colleague notified me this discussion, which is not certainly the first on the web about this old and hot topic.

  
  
  
  

  After reading some posts, the first thing I did was looking for the "?R" string in this thread. The second was to search about "recursion".
  No, holy cow, no match found.
  Since nobody mentioned the main mechanism a parser is built onto, I was soon aware that nobody got the point.

  
  
  
  

  If an (x)HTML parser needs recursion, a RegExp parser without recursion is not enough for the purpose. It's a simple construct.

  
  
  
  

  The black art of RegExp is hard to master, so maybe there are further possibilities we left out while trying and testing our personal solution to capture the whole web in one hand... Well, I am sure about it :)

  
  
  
  

  Here's the magic pattern:

  
  

  $pattern = "/<([w]+)([^>]*?)(([s]*/>)|(>((([^<]*?|<!--.*?-->)|(?R))*)</1[s]*>))/s";


  
  
  
  

  Just try it.
  It's written as a PHP string, so the "s" modifier makes classes include newlines.
  Here's a sample note on the PHP manual I wrote on January: Reference

  
  
  
  

  (Take care, in that note I wrongly used the "m" modifier; it should be erased, notwithstanding it is discarded by the RegExp engine, since no ^ or $ anchorage was used).

  
  
  
  

  Now, we could speak about the limits of this method from a more informed point of view:

  
  
  
  

  Anyhow it is only a RegExp pattern, but it discloses the possibility to develop of a lot of powerful implementations.
  I wrote this pattern to power the recursive descent parser of a template engine I built in my framework, and performances are really great, both in execution times or in memory usage (nothing to do with other template engines which use the same syntax).

  
  
  
  
  
  

  
  I'll put this in the "Regex which doesn't allow greater-than in attributes" bin. Check it against <input value="is 5 > 3?" />
  – Gareth
  Jul 5 '10 at 16:24
  

  
  
  
  
  
  
  
  

  
  If you put something like that in production code, you would likely be shot by the maintainer. A jury would never convict him.
  – aehiilrs
  Jul 5 '10 at 16:33
  

  
  
  
  
  
  
  
  

  
  Regular expressions can't work because by definition they are not recursive. Adding a recursive operator to regular expressions basically makes a CFG only with poorer syntax. Why not use something designed to be recursive in the first place rather than violently insert recursion into something already overflowing with extraneous functionality?
  – Welbog
  Jul 6 '10 at 18:38
  

  
  
  
  
  
  
  
  

  
  My objection isn't one of functionality it is one of time invested. The problem with RegEx is that by the time you post the cutsey little one liners it appears that you did something more efficiently ("See one line of code!"). And of course no one mentions the half hour (or 3) that they spent with their cheat-sheet and (hopefully) testing every possible permutation of input. And once you get past all that when the maintainer goes to figure out or validate the code they can't just look at it and see that it is right. The have to dissect the expression and essentially retest it all over again...
  – Oorang
  Jul 10 '10 at 15:11
  

  
  
  
  
  
  
  
  

  
  ... to know that it is good. And that will happen even with people who are good with regex. And honestly I suspect that overwhelming majority of people won't know it well. So you take one of the most notorious maintenance nightmares and combine it with recursion which is the other maintenance nightmare and I think to myself what I really need on my project is someone a little less clever. The goal is to write code that bad programmers can maintain without breaking the code base. I know it galls to code to the least common denominator. But hiring excellent talent is hard, and you often...
  – Oorang
  Jul 10 '10 at 15:17
  

  
  
  
  
  
  

  As many people have already pointed out, HTML is not a regular language which can make it very difficult to parse. My solution to this is to turn it into a regular language using a tidy program and then to use an XML parser to consume the results. There are a lot of good options for this. My program is written using Java with the jtidy library to turn the HTML into XML and then Jaxen to xpath into the result.

  
  
  
  
  
  

  
  That was helpful, thanks
  – Khemraj
  Oct 27 '17 at 21:16
  

  
  
  
  

  <s*(w+)[^/>]*>


  
  
  
  

  The parts explained:

  
  
  
  

  <: starting character

  
  

  <

  
  
  
  

  s*: it may have whitespaces before tag name (ugly but possible).

  
  

  s*

  
  
  
  

  (w+): tags can contain letters and numbers (h1). Well, w also matches '_', but it does not hurt I guess. If curious use ([a-zA-Z0-9]+) instead.

  
  

  (w+)

  
  

  w

  
  
  
  

  [^/>]*: anything except > and / until closing >

  
  

  [^/>]*

  
  

  >

  
  

  /

  
  

  >

  
  
  
  

  >: closing >

  
  

  >

  
  

  >

  
  
  
  

  And to fellows who underestimate regular expressions saying they are only as powerful as regular languages:

  
  
  
  

  aⁿbaⁿbaⁿ which is not regular and not even context free, can be matched with ^(a+)b1b1$

  
  

  ^(a+)b1b1$

  
  
  
  

  Backreferencing FTW!

  
  
  
  
  
  

  
  @GlitchMr, that was his point. Modern regular expressions are not technically regular, nor is there any reason for them to be.
  – alanaktion
  Feb 2 '13 at 15:45
  

  
  
  
  
  
  
  
  

  
  @alanaktion: The "modern" regular expressions (read: with Perl extensions) cannot match within O(MN) (M being regular expression length, N being text length). Backreferences are one of causes of that. The implementation in awk doesn't have backreferences and matches everything within O(MN) time.
  – Konrad Borowski
  Feb 14 '13 at 16:52
  

  
  
  
  

  O(MN)

  
  

  O(MN)

  
  
  
  

  I recently wrote an HTML sanitizer in Java. It is based on a mixed approach of regular expressions and Java code. Personally I hate regular expressions and its folly (readability, maintainability, etc.), but if you reduce the scope of its applications it may fit your needs. Anyway, my sanitizer uses a white list for HTML tags and a black list for some style attributes.

  
  
  
  

  For your convenience I have set up a playground so you can test if the code matches your requirements: playground and Java code. Your feedback will be appreciated.

  
  
  
  

  There is a small article describing this work on my blog: http://roberto.open-lab.com

  
  
  
  
  
  

  
  Your links are dead and this isn't really providing any useful answer.
  – kenorb
  May 19 '15 at 16:23
  
  

  
  
  
  
  
  

  It seems to me you're trying to match tags without a "/" at the end. Try this:

  
  

  <([a-zA-Z][a-zA-Z0-9]*)[^>]*(?<!/)>


  
  
  
  
  
  

  
  This does not work. For the input '<x a="<b>"/><y>' the matches are x and y, although x is terminated.
  – ceving
  May 4 '11 at 16:33
  

  
  
  
  
  
  

  If you're simply trying to find those tags (without ambitions of parsing) try this regular expression:

  
  

  /<[^/]*?>/g


  
  
  
  

  I wrote it in 30 seconds, and tested here:
  http://gskinner.com/RegExr/

  
  
  
  

  It matches the types of tags you mentioned, while ignoring the types you said you wanted to ignore.

  
  
  
  
  
  

  
  I think you mean /> instead of >.
  – Justin Morgan
  Dec 19 '14 at 17:36
  
  

  
  
  
  

  />

  
  

  >

  
  
  
  
  
  

  
  No, just > is what I meant; I never meant to edit the regular expression of my original post.
  – Lonnie Best
  May 29 '16 at 6:38
  

  
  
  
  

  >

  
  
  
  
  
  

  
  FYI, you don't need to escape angle brackets. Of course, it does no harm to escape them anyway, but look at the confusion you could have avoided. ;)
  – Alan Moore
  May 29 '16 at 7:47
  

  
  
  
  
  
  
  
  

  
  I sometimes escape unnecessarily when I'm unsure if something is special character or not. I've edited the answer; it works the same but more concise.
  – Lonnie Best
  May 31 '16 at 7:23
  

  
  
  
  
  
  
  
  

  
  Looking at this now, I don't know why I thought you meant /, since that would do the exact opposite of the requirements. Maybe I thought you were offering a negative filter pattern.
  – Justin Morgan
  Jun 1 '16 at 19:14
  

  
  
  
  

  /

  
  
  
  

  Although it's not suitable and effective to use regular expressions for that purpose sometimes regular expressions provide quick solutions for simple match problems and in my view it's not that horrbile to use regular expressions for trivial works.

  
  
  
  

  There is a definitive blog post about matching innermost HTML elements written by Steven Levithan.

  
  
  
  

  
  Thank you for your interest in this question.
  Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
  
  
  Would you like to answer one of these unanswered questions instead?

  y6,9 WsQWjR65Q,j6bYyb8hSx 9DV4b W3oCBsY0,M LcKEdx46QCGHe904,n IVuLmq v5Z
- July 13, 2018
html regex xhtml
Related Preview


How to add margin between elements ...


insert table inside div between two...


Diving into span data - parsing HTM...


How to count words in MySQL / regul...


Reference - What does this regex me...
• Get link
• Facebook
• X
• Pinterest
• Email
• Other Apps