Linux Certif

Toute la documentation sur la certification Linux LPI

clive

NAME

clive - the non-interactive video extraction utility

SYNOPSIS

clive [option]... [URL]...

DESCRIPTION

clive is a command line utility for extracting videos from Youtube and other video sharing Web sites. It was originally written to bypass the Adobe Flash requirement needed to view the hosted videos.

clive can be non-interactive. This means it can work in the background while the user is not logged on, allowing the user to start an extraction and disconnect from the system and letting clive finish the work.

clive was influenced by the GNU wget(1) utility. Kudos to the wget team for their original work.

OPTION SYNTAX

You may freely mix different option styles, or specify options after the command-line arguments. For example:
  % clive -c --format=mp4 URL -n

You may also put several options together that do not require arguments. For example:
  % clive -pcn URL

Which is equivalent to:
  % clive -p -c -n URL

OPTIONS

-h --help

Print this help and exit.

-v --version

Print version and exit.

--hosts

Print supported hosts and exit.

-b --background

Go to background immediately after startup. If no output file is specified using the --output or --append, the output is redirected to clive-log file. Implies --progress=dot.

-e --emit-csv

Do not extract any videos, instead print the video details as CSV to stdout. Ignores the --quiet option for CSV.

-E --emit-xml

Equivalent to --emit-csv in every way but the output is XML. Negates --emit-csv.

-V --clivepass=path

path to the clivepass(1) utility. If CLIVEPASS_PATH environment variable is set, it will will used.

HTTP options

-U --agent=string

Identify as string to the HTTP server. Defaults is ``Mozilla/5.0''.

-y --proxy=address

Use address for HTTP proxy, e.g. http://foo:1234. If undefined, uses the http_proxy environment setting if present.

-X --no-proxy

Do not use HTTP proxy.

Cache Options

-R --renew

Renew cache entries for the re-visited URLs. See ``CACHE'' in the manual page.

-s --show

Print all cache entries to standard output.

-H --show-format=string

Format printed entries using the string. Default is '%D: ``%t'' | %mMB'. The following specifiers are supported:
  %t = video page title
  %i = video id
  %l = video file length (bytes)
  %u = video page url
  %x = video extraction url
  %D = video extraction date
  %T = video extraction time
  %S = video extraction timestamp (same as: %D %T)

-g --grep=pattern

Grep cache entries for pattern. Matching is not restricted to any specific cache entry field (e.g. title or id).
  % clive --grep=git
  % clive --grep=^git --ignore-case

The matched entries are then extracted. To only show the matches, use the --show option. For example:
  % clive -sg ^git

-i --ignore-case

Causes clive to ignore case-differences with --grep.

-D --delete

Delete grep-matched entries from the cache. If used with the --show option, clive will prompt before deleting, e.g.:
  % clive -siDg ^git

-C --clear

Clear cache and exit.

Logging and Input Options

-o --output=logfile

Log all messages to logfile. The messages are normally reported to stdout and stderr.

-a --append=logfile

Append to logfile. This is the same as --output but appends to logfile instead of overwriting it. If the logfile does not exist, it is created.

-d --debug

Causes the program to print Curl debug messages.

-q --quiet

Turn off all output.

-r --recall

Recall the last URL batch from the previous runtime. Note that clive overwrites this information at every runtime. See also ``CACHE''.

-T --savebatch=file

Save current URL batch to file.

-p --paste

Paste input from clipboard. The pasted URLs are expected to be separated with newlines.

--stderr

Redirect all input to stderr, even if no TTY is available.

Download Options

-O --output-video=file

Write video to file.

-n --no-extract

Do not actually extract any videos.

-c --continue

Continue extraction of a partially downloaded file. Note that this works only with HTTP servers that support the ``Range'' header. Ignored unless localfile < remotefile.

The ``requested range was not delivered'' error typically implies that the host does not allow continuing partially extracted video files. You will see this error if you attempt to continue a partially downloaded flv video from Youtube, for example.

-W --overwrite

Causes clive to overwrite already existing video files. By default clive appends a numeric suffix to the output filename.

-G --progress=type

Select the type of the progress indicator. Accepted types are ``bar'', ``dot'' and ``none''.

The ``bar'' indicator is used by default. It spews out some details indicating the status of retrieval. If the output is not a TTY, the ``dot'' bar will be used instead.

Use --progress=dot to switch to the ``dot'' display. It traces the retrieval by printing dots on the screen, each dot representing a fixed amount of downloaded data.

When using the dotted retrieval, you may also set the style by specifying the type as dot:style.
  --progress=dot:default
    each dot 1K, 10-dots clusters, 50-dots lines
  --progress=dot:binary
    each dot  8K, 16-dots clusters, 48-dots lines
  --progress=dot:mega
    each dot 64K,  8-dots clusters, 48-dots lines

Progress indication can be disabled completely by setting type to ``none''.

-u --youtube-user=username

username to log into Youtube. Typically needed only if you are planning to extract videos flagged as mature-content.

-t --youtube-pass=password

password to log into Youtube. If password is ``-'', causes clive to retrieve the login password from the clivepass(1) utility. See ``EXAMPLES''.

-L --no-login

Do not log into Youtube.

-S --savedir=dir

Save extracted videos to dir.

-f --format=format

Extract format of the video. All supported websites have at least the flv format available. See ``FORMATS'' for more.

-l --cclass=character-class

Use character-class to filter the video page titles. Default is \w. The filtered video page titles are used to name the extracted videos.

-N --filename-format=string

Use string to construct the name of the extracted video. Default is ``%t_%d_%i.%s''. The following specifiers are supported:
  %t = video title (after applying --cclass)
  %i = video id
  %d = video domain
  %s = video suffix
  %D = current date
  %T = current time
  %S = timestamp (same as %D %T)

-x --exec=command;

Execute command. Optional arguments may be passed to the command. The expression must be terminated by a semicolon (``;''). If the specifier ``%i'' appears anywhere in command name or the arguments it is replaced by the pathname of the extracted video file.

Example:
  % clive --exec=``mplayer %i;'' URL

-x, --exec=command+

Same as --exec, except that ``%i'' is replaced with as many pathnames as possible for the invocation of command.

--stream-exec=command

Execute command when file transfer progress reaches percent as defined with the --stream option. The ``%i'' specifier for video pathname is expected to appear somewhere in the command expression.

Unlike the --exec option which is executed subsequently after each file transfer, clive forks the streaming process and continues with the file transfer --- and will not continue to download another file until the child process terminates.

It should also be noted that clive makes no effort to check whether there is enough data to be streamed. It assumes the user knows how much data needs to be buffered before starting the child process. Similarly, if the transfer rate drops significantly after starting the process and it runs out of data, clive simply ignores that.

Some players and formats may not work properly with this feature. Try mplayer(1) and choose flv if you are unsure.

--stream=percent

Execute the command defined with the --stream-exec option when the file transfer progress reaches percent.

Example:
  % clive --stream-exec=``mplayer -really-quiet %i'' --stream=20 URL

--limit-rate=amount

Limit video download rate to amount KB/s.

EXAMPLES

Typical use:

% clive ``http://youtube.com/watch?v=3HD220e0bx4''

Multiple video URLs:

% cat >> url.lst
  http://en.sevenload.com/videos/IUL3gda-Funny-Football-Clips
  http://youtube.com/watch?v=3HD220e0bx4
  http://break.com/index/beach-tackle-whip-lash.html
  http://www.liveleak.com/view?i=704_1228511265

% cat url.lst | clive

Combine input from multiple sources:

% clive -p URL URL2

Recall the last URL batch, extract mp4 video format:

% clive -rf mp4

Grep for the pattern in cache, extract matched videos:

% clive -g 3HD220e0bx4

Same as above but delete the matched entries from cache:

% clive -iDg ^3hd2

Note the use of -i (--ignore-case).

Print cache contents to stdout:

% clive -s

Combine matching and printing cache entries:

% clive -sig ^3hd2

Note that -s negates video extraction.

Match, redirect output to my.log, go to background:

% clive -big ^3hd2 -o my.log

Extract video from URL, re-encode audio to mp3:

% clive --exec=``ffmpeg -y -i %i %i.mpg;'' URL

Login to Youtube to extract a video:

% clive -u myusername -t mypassword URL

This isn't typically needed unless you are trying to extract videos flagged as mature content.

Use protected login password:

Otherwise same as above but the login info is saved to a clivepass password repository file. The saved passwords are encrypted.

Note that you will need to install the clive-utils for this. See the ``OTHER'' section for details.

Create new password repository:
  % clivepass --create

Add new login name ``myusername'':
  % clivepass --add myusername

Use the created login info from clive:
  % clive -u myusername -t - -V /usr/bin/clivepass URL

This causes clive to request for the login password for ``myusername'' from clivepass(1). The password is then used to login to Youtube before extracting the video from the URL.

FORMATS

clive defaults to extract the flv format unless the --format option is used. The requested format may not always be available. The server typically returns one of the HTTP/40x errors if the requested format is not available.

The quality of the video depends on the uploaded video quality. Each website typically recompresses the uploaded videos to 320x240 resolution (sometimes higher). Because this varies per video and website, you should treat the list below only as a rough reference of what formats should be available.

www.youtube.com

www.last.fm

Formats: (flv|mp4|mp4_hd|3gpp|xflv) Continue: Yes (except flv)

mp4 and 3gpp formats are typically, or will soon become, available. xflv appears to be very rare. Videos dating back to 2006 may only be available as flv. The high-definition mp4_hd format is currently available for a limited number of videos, which are usually explicitly marked as ``HD'' or ``720p''.

Some of the videos displayed on last.fm site are actually Youtube videos. You can pass the lastfm'ed video URLs to clive.

video.google.com

Formats: (flv|mp4) Continue: Only mp4

The mp4 may not always be available. GoogleVideo mp4 are encoded using a different codec than Youtube mp4 videos. You can typically --stream-exec Youtube mp4 videos but this is not case with GoogleVideo mp4 videos. If you find a way, drop an email.

www.sevenload.com

Formats: flv Continue: Yes

www.break.com

Formats: flv Continue: Yes

www.liveleak.com

Formats: flv Continue: Yes

www.evisor.tv

Formats: flv Continue: Yes

www.dailymotion.com

Formats: (spark|spak-mini|vp6-hq|vp6-hd|vp6|h264) Continue: Yes

spark is the ``regular'' 320x240 flv. clive defaults to spark if requested format is not available.

tv.cctv.com

Formats: flv Continue: Yes

CACHE

The cache has two purposes:

1.

Keep reusable video data available for fast re-extraction.

2.

Keep a record of extracted videos.

Each cache entry contains information about a video, including, but not limited to, page title, file length and extraction URL. You can use the --grep option to match these details when you are searching for videos from the cache. See the ``EXAMPLES'' section for examples of how to do this.

Some entries may need to be renewed from time to time as certain websites have their extraction URLs expire after awhile. Youtube is an example of this. Youtube servers usually return the HTTP/410 error if the extraction URL has expired. You can use the --renew option to fix this.

Note that clive renews the cache entry automatically if the video format does not match the cached video format string. This is done for two reasons:

1.

Cached video extraction URL points to a different location

2.

Cached video file length is likely to be incorrect

UNICODE

The author has had some success with uxterm(1) previously. Truth to be told, this is by far one of the least favorite topics of mine. I blame Python (partly) for this. See your local unicode-expert on this.

My two cents: As long as the terminal is unicode capable and the used terminal font supports the characters, this shouldn't be a problem. You may want to use --cclass=. when you are running tests. That disables video title filtering.

FILES

clive searches the ~/.config/clive directory for the config file. You can override this by setting the CLIVE_HOME environment variable.

~/.config/clive/config

Configuration file for clive.

~/.config/clive/cache

Contains the cache entries of the visited URLs. A Berkeley DB (Hash) file.

~/.config/clive/recall

Contains the last URL batch. You can --recall this file.

EXIT STATUS

clive exits 0 on success, and >0 if an error occurs.

SEE ALSO

clivefeed(1) clivescan(1) clivepass(1)

OTHER

Homepage:
  <http://clive.sourceforge.net/>

Project page:
  <http://code.google.com/p/clive/>

Development repository can be cloned with:
  % git clone git://repo.or.cz/clive.git

For release announcements, subscribe to the project at:
  <http://freshmeat.net/projects/clive/>

Additional utilities (clive-utils):
  <http://code.google.com/p/clive-utils/>

AUTHOR

Written by Toni Gundogdu <legatvs@gmail.com>.