URL Parser - `libvcs.url`¶

Parse VCS URLs into typed, editable structures — urllib.parse for git, Mercurial, and Subversion. You validate a URL string, read its parts back as dataclasses fields (hostname, path, rev), change any of them, and export a form the VCS binary accepts.

The common URL shapes work out of the box; when your host or shorthand isn’t covered, you can add your own rules. For a guided tour, start with URL Parsing.

Modules¶

Git URLs

Parse and validate Git repository URLs (HTTPS, SSH, SCP).

Git URL Parser - libvcs.url.git

SVN URLs

Parse Subversion repository URLs.

SVN URL Parser - libvcs.url.svn

Hg URLs

Parse Mercurial repository URLs.

Mercurial URL Parser - libvcs.url.hg

Base

Abstract base classes for URL parsing.

Framework: Add and extend URL parsers - libvcs.url.base

Registry

URL matcher registration and lookup.

VCS Detection - libvcs.url.registry

Constants

Shared regex patterns and URL constants.

Constants - libvcs.url.constants

Validate and detect VCS URLs¶

Check whether a string is a URL the VCS recognizes:

git

libvcs.url.git.GitURL.is_valid()

>>> from libvcs.url.git import GitURL

>>> GitURL.is_valid(url='https://github.com/vcs-python/libvcs.git')
True

>>> from libvcs.url.git import GitURL

>>> GitURL.is_valid(url='[email protected]:vcs-python/libvcs.git')
True

libvcs.url.hg.HgURL.is_valid()

>>> from libvcs.url.hg import HgURL

>>> HgURL.is_valid(url='https://hg.mozilla.org/mozilla-central/mozilla-central')
True

>>> from libvcs.url.hg import HgURL

>>> HgURL.is_valid(url='[email protected]:MyProject/project')
True

svn

libvcs.url.svn.SvnURL.is_valid()

>>> from libvcs.url.svn import SvnURL

>>> SvnURL.is_valid(
... url='https://svn.project.org/project-central/project-central')
True

>>> from libvcs.url.svn import SvnURL

>>> SvnURL.is_valid(url='[email protected]:MyProject/project')
True

Parse VCS URLs¶

Turn a URL string into a typed structure with named fields — compare to urllib.parse.ParseResult:

git

libvcs.url.git.GitURL

>>> from libvcs.url.git import GitURL

>>> GitURL(url='[email protected]:vcs-python/libvcs.git')
GitURL([email protected]:vcs-python/libvcs.git,
        user=git,
        hostname=github.com,
        path=vcs-python/libvcs,
        suffix=.git,
        rule=core-git-scp)

libvcs.url.hg.HgURL

>>> from libvcs.url.hg import HgURL

>>> HgURL(
...     url="http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin")
HgURL(url=http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin,
        scheme=http,
        hostname=hugin.hg.sourceforge.net,
        port=8000,
        path=hgroot/hugin/hugin,
        rule=core-hg)

svn

libvcs.url.svn.SvnURL

>>> from libvcs.url.svn import SvnURL

>>> SvnURL(
...     url='svn+ssh://svn.debian.org/svn/aliothproj/path/in/project/repository')
SvnURL(url=svn+ssh://svn.debian.org/svn/aliothproj/path/in/project/repository,
       scheme=svn+ssh,
       hostname=svn.debian.org,
       path=svn/aliothproj/path/in/project/repository,
       rule=pip-url)

Export usable URLs¶

pip knows what a certain URL string means, but git clone won’t. This works great with pip:

$ pip install git+https://github.com/django/[email protected]
...
Successfully installed Django-3.2

but git clone can’t use that URL:

$ git clone git+https://github.com/django/[email protected]
Cloning into '[email protected]'...
fatal: Unable to find remote helper for 'git+https'

It needs something like this:

$ git clone https://github.com/django/django.git --branch 3.2

That translation is why parsing returns a structure — GitURL — rather than a string. As with urllib.parse.ParseResult, you inspect or replace individual fields (swap just the hostname, drop the scheme), then call to_url() when you finally need a string the VCS accepts. The same structure covers the popular hosts out of the box and takes custom rules for everything else.

Scope¶

Out of the box¶

libvcs parses package-like URLs, e.g.

Vanilla VCS URLs
- any URL supported by the VCS binary, e.g. git(1), svn(1), hg(1).
pip-style urls [1]
- branches
- tags
NPM-style urls[2]
- branches
- tags

Extendability¶

Patterns can be registered. Similar behavior exists in urllib.parse (undocumented).

Any formats not covered by the stock rules
Custom URLs
- For orgs on GitHub or GitLab, e.g.:
  - python:mypy -> git@github.com:python/mypy.git
  - inkscape:inkscape -> git@gitlab.com:inkscape/inkscape.git
- For out of domain trackers, e.g.
  
  Direct to site:
  - cb:vcs-python/libvcs -> https://codeberg.org/vcs-python/libvcs
  - kde:plasma/plasma-sdk -> git@invent.kde.org:plasma/plasma-sdk.git
    
    Aside: Note KDE’s git docs use of url.<base>.insteadOf and url.<base>.pushInsteadOf
  Direct to site + org / group:
  - gnome:gedit -> git@gitlab.gnome.org:GNOME/gedit.git
  - openstack:openstack -> https://opendev.org/openstack/openstack.git
  - mozilla:central -> https://hg.mozilla.org/mozilla-central/

From there, GitURL can be used downstream directly by other projects: libvcs’s own Commands - libvcs.cmd and sync layers, as well as vcspull configurations, detect and accept these URL patterns.

How matching resolves¶

When a rule matches, its defaults fill in the groups the pattern didn’t capture — a github: prefix implies hostname=github.com. When several rules could match, higher weights are checked first; the first rule whose pattern produces a valid match wins.