Skip to content
Snippets Groups Projects
Commit 1f49493f authored by Adam J. Stewart's avatar Adam J. Stewart Committed by Todd Gamblin
Browse files

Major improvements to spack create (#2707) 

* Initial changes to spack create command

* Get 'spack create <url>' working again

* Simplify call to BuildSystemGuesser

* More verbose output of spack create

* Remove duplicated code from spack create and spack checksum

* Add better documentation to spack create docstrings

* Fix pluralization bug

* Flake8

* Update documentation on spack create and deprecate spack edit --force

* Make it more obvious when we are renaming a package

* Further deprecate spack edit --force

* Fix unit tests

* Rename default template to generic template

* Don't add automake/autoconf deps to Autotools packages

* Remove changes to default $EDITOR

* Completely remove all traces of spack edit --force

* Remove grammar changes to make the PR easier to review
parent beafcfd3
Branches
Tags
No related merge requests found
Expand all Hide whitespace changes
Inline Side-by-side
lib/spack/docs/packaging_guide.rst
+ 140
− 157
View file @ 1f49493f
...@@ -17,7 +17,7 @@ There are two key parts of Spack: ...@@ -17,7 +17,7 @@ There are two key parts of Spack:
software according to a spec. software according to a spec.
Specs allow a user to describe a *particular* build in a way that a Specs allow a user to describe a *particular* build in a way that a
package author can understand. Packages allow a the packager to package author can understand. Packages allow the packager to
encapsulate the build logic for different versions, compilers, encapsulate the build logic for different versions, compilers,
options, platforms, and dependency combinations in one place. options, platforms, and dependency combinations in one place.
Essentially, a package translates a spec into build logic. Essentially, a package translates a spec into build logic.
...@@ -40,87 +40,68 @@ Creating & editing packages ...@@ -40,87 +40,68 @@ Creating & editing packages
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^
The ``spack create`` command creates a directory with the package name and The ``spack create`` command creates a directory with the package name and
generates a ``package.py`` file with a boilerplate package template from a URL. generates a ``package.py`` file with a boilerplate package template. If given
The URL should point to a tarball or other software archive. In most cases, a URL pointing to a tarball or other software archive, ``spack create`` is
``spack create`` plus a few modifications is all you need to get a package smart enough to determine basic information about the package, including its name
working. and build system. In most cases, ``spack create`` plus a few modifications is
all you need to get a package working.
Here's an example: Here's an example:
.. code-block:: console .. code-block:: console
$ spack create http://www.cmake.org/files/v2.8/cmake-2.8.12.1.tar.gz $ spack create https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2
Spack examines the tarball URL and tries to figure out the name of the package Spack examines the tarball URL and tries to figure out the name of the package
to be created. Once the name is determined a directory in the appropriate to be created. If the name contains uppercase letters, these are automatically
repository is created with that name. Spack prefers, but does not require, that converted to lowercase. If the name contains underscores or periods, these are
names be lower case so the directory name will be lower case when ``spack automatically converted to dashes.
create`` generates it. In cases where it is desired to have mixed case or upper
case simply rename the directory. Spack also tries to determine what version Spack also searches for *additional* versions located in the same directory of
strings look like for this package. Using this information, it will try to find the website. Spack prompts you to tell you how many versions it found and asks
*additional* versions by spidering the package's webpage. If it finds multiple you how many you would like to download and checksum:
versions, Spack prompts you to tell it how many versions you want to download
and checksum:
.. code-block:: console .. code-block:: console
$ spack create http://www.cmake.org/files/v2.8/cmake-2.8.12.1.tar.gz $ spack create https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2
==> This looks like a URL for cmake version 2.8.12.1. ==> This looks like a URL for gmp
==> Creating template for package cmake ==> Found 16 versions of gmp:
==> Found 18 versions of cmake.
2.8.12.1 http://www.cmake.org/files/v2.8/cmake-2.8.12.1.tar.gz 6.1.2 https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2
2.8.12 http://www.cmake.org/files/v2.8/cmake-2.8.12.tar.gz 6.1.1 https://gmplib.org/download/gmp/gmp-6.1.1.tar.bz2
2.8.11.2 http://www.cmake.org/files/v2.8/cmake-2.8.11.2.tar.gz 6.1.0 https://gmplib.org/download/gmp/gmp-6.1.0.tar.bz2
... ...
2.8.0 http://www.cmake.org/files/v2.8/cmake-2.8.0.tar.gz 5.0.0 https://gmplib.org/download/gmp/gmp-5.0.0.tar.bz2
Include how many checksums in the package file? (default is 5, q to abort) How many would you like to checksum? (default is 1, q to abort)
Spack will automatically download the number of tarballs you specify Spack will automatically download the number of tarballs you specify
(starting with the most recent) and checksum each of them. (starting with the most recent) and checksum each of them.
You do not *have* to download all of the versions up front. You can You do not *have* to download all of the versions up front. You can
always choose to download just one tarball initially, and run always choose to download just one tarball initially, and run
:ref:`cmd-spack-checksum` later if you need more. :ref:`cmd-spack-checksum` later if you need more versions.
.. note::
If ``spack create`` fails to detect the package name correctly,
you can try supplying it yourself, e.g.:
.. code-block:: console
$ spack create --name cmake http://www.cmake.org/files/v2.8/cmake-2.8.12.1.tar.gz
If it fails entirely, you can get minimal boilerplate by using
:ref:`spack edit --force <spack-edit-f>`, or you can manually create a
directory and ``package.py`` file for the package in
``var/spack/repos/builtin/packages``, or within your own :ref:`package
repository <repositories>`.
.. note::
Spack can fetch packages from source code repositories, but,
``spack create`` will *not* currently create a boilerplate package
from a repository URL. You will need to use :ref:`spack edit --force <spack-edit-f>`
and manually edit the ``version()`` directives to fetch from a
repo. See :ref:`vcs-fetch` for details.
Let's say you download 3 tarballs: Let's say you download 3 tarballs:
.. code-block:: none .. code-block:: console
Include how many checksums in the package file? (default is 5, q to abort) 3
==> Downloading...
==> Fetching http://www.cmake.org/files/v2.8/cmake-2.8.12.1.tar.gz
###################################################################### 98.6%
==> Fetching http://www.cmake.org/files/v2.8/cmake-2.8.12.tar.gz
##################################################################### 96.7%
==> Fetching http://www.cmake.org/files/v2.8/cmake-2.8.11.2.tar.gz
#################################################################### 95.2%
Now Spack generates boilerplate code and opens a new ``package.py`` How many would you like to checksum? (default is 1, q to abort) 3
file in your favorite ``$EDITOR``: ==> Downloading...
==> Fetching https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2
######################################################################## 100.0%
==> Fetching https://gmplib.org/download/gmp/gmp-6.1.1.tar.bz2
######################################################################## 100.0%
==> Fetching https://gmplib.org/download/gmp/gmp-6.1.0.tar.bz2
######################################################################## 100.0%
==> Checksummed 3 versions of gmp:
==> This package looks like it uses the autotools build system
==> Created template for gmp package
==> Created package file: /Users/Adam/spack/var/spack/repos/builtin/packages/gmp/package.py
Spack automatically creates a directory in the appropriate repository,
generates a boilerplate template for your package, and opens up the new
``package.py`` in your favorite ``$EDITOR``:
.. code-block:: python .. code-block:: python
:linenos: :linenos:
...@@ -130,11 +111,11 @@ file in your favorite ``$EDITOR``: ...@@ -130,11 +111,11 @@ file in your favorite ``$EDITOR``:
# next to all the things you'll want to change. Once you've handled # next to all the things you'll want to change. Once you've handled
# them, you can save this file and test your package like this: # them, you can save this file and test your package like this:
# #
# spack install cmake # spack install gmp
# #
# You can edit this file again by typing: # You can edit this file again by typing:
# #
# spack edit cmake # spack edit gmp
# #
# See the Spack documentation for more information on packaging. # See the Spack documentation for more information on packaging.
# If you submit this package back to Spack as a pull request, # If you submit this package back to Spack as a pull request,
...@@ -143,33 +124,46 @@ file in your favorite ``$EDITOR``: ...@@ -143,33 +124,46 @@ file in your favorite ``$EDITOR``:
from spack import * from spack import *
class Cmake(Package): class Gmp(AutotoolsPackage):
"""FIXME: Put a proper description of your package here.""" """FIXME: Put a proper description of your package here."""
# FIXME: Add a proper url for your package's homepage here. # FIXME: Add a proper url for your package's homepage here.
homepage = "http://www.example.com" homepage = "http://www.example.com"
url = "http://www.cmake.org/files/v2.8/cmake-2.8.12.1.tar.gz" url = "https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2"
version('2.8.12.1', '9d38cd4e2c94c3cea97d0e2924814acc')
version('2.8.12', '105bc6d21cc2e9b6aff901e43c53afea')
version('2.8.11.2', '6f5d7b8e7534a5d9e1a7664ba63cf882')
# FIXME: Add dependencies if this package requires them. version('6.1.2', '8ddbb26dc3bd4e2302984debba1406a5')
# depends_on("foo") version('6.1.1', '4c175f86e11eb32d8bf9872ca3a8e11d')
version('6.1.0', '86ee6e54ebfc4a90b643a65e402c4048')
def install(self, spec, prefix): # FIXME: Add dependencies if required.
# FIXME: Modify the configure line to suit your build system here. # depends_on('foo')
configure("--prefix=" + prefix)
# FIXME: Add logic to build and install here def configure_args(self):
make() # FIXME: Add arguments other than --prefix
make("install") # FIXME: If not needed delete the function
args = []
return args
The tedious stuff (creating the class, checksumming archives) has been The tedious stuff (creating the class, checksumming archives) has been
done for you. done for you. You'll notice that ``spack create`` correctly detected that
``gmp`` uses the Autotools build system. It created a new ``Gmp`` package
that subclasses the ``AutotoolsPackage`` base class. This base class
provides basic installation methods common to all Autotools packages:
.. code-block:: bash
./configure --prefix=/path/to/installation/directory
make
make check
make install
For most Autotools packages, this is sufficient. If you need to add
additional arguments to the ``./configure`` call, add them via the
``configure_args`` function.
In the generated package, the download ``url`` attribute is already In the generated package, the download ``url`` attribute is already
set. All the things you still need to change are marked with set. All the things you still need to change are marked with
``FIXME`` labels. You can delete the commented instructions between ``FIXME`` labels. You can delete the commented instructions between
the license and the first import statement after reading them. the license and the first import statement after reading them.
The rest of the tasks you need to do are as follows: The rest of the tasks you need to do are as follows:
...@@ -177,105 +171,96 @@ The rest of the tasks you need to do are as follows: ...@@ -177,105 +171,96 @@ The rest of the tasks you need to do are as follows:
#. Add a description. #. Add a description.
Immediately inside the package class is a *docstring* in Immediately inside the package class is a *docstring* in
triple-quotes (``"""``). It's used to generate the description triple-quotes (``"""``). It is used to generate the description
shown when users run ``spack info``. shown when users run ``spack info``.
#. Change the ``homepage`` to a useful URL. #. Change the ``homepage`` to a useful URL.
The ``homepage`` is displayed when users run ``spack info`` so The ``homepage`` is displayed when users run ``spack info`` so
that they can learn about packages. that they can learn more about your package.
#. Add ``depends_on()`` calls for the package's dependencies. #. Add ``depends_on()`` calls for the package's dependencies.
``depends_on`` tells Spack that other packages need to be built ``depends_on`` tells Spack that other packages need to be built
and installed before this one. See :ref:`dependencies`. and installed before this one. See :ref:`dependencies`.
#. Get the ``install()`` method working. #. Get the installation working.
The ``install()`` method implements the logic to build a Your new package may require specific flags during ``configure``.
package. The code should look familiar; it is designed to look These can be added via ``configure_args``. Specifics will differ
like a shell script. Specifics will differ depending on the package, depending on the package and its build system.
and :ref:`implementing the install method <install-method>` is :ref:`Implementing the install method <install-method>` is
covered in detail later. covered in detail later.
Before going into details, we'll cover a few more basics. Passing a URL to ``spack create`` is a convenient and easy way to get
a basic package template, but what if your software is licensed and
.. _cmd-spack-edit: cannot be downloaded from a URL? You can still create a boilerplate
``package.py`` by telling ``spack create`` what name you want to use:
^^^^^^^^^^^^^^
``spack edit``
^^^^^^^^^^^^^^
One of the easiest ways to learn to write packages is to look at
existing ones. You can edit a package file by name with the ``spack
edit`` command:
.. code-block:: console .. code-block:: console
$ spack edit cmake $ spack create --name intel
So, if you used ``spack create`` to create a package, then saved and This will create a simple ``intel`` package with an ``install()``
closed the resulting file, you can get back to it with ``spack edit``. method that you can craft to install your package.
The ``cmake`` package actually lives in
``$SPACK_ROOT/var/spack/repos/builtin/packages/cmake/package.py``,
but this provides a much simpler shortcut and saves you the trouble
of typing the full path.
If you try to edit a package that doesn't exist, Spack will recommend What if ``spack create <url>`` guessed the wrong name or build system?
using ``spack create`` or ``spack edit --force``: For example, if your package uses the Autotools build system but does
not come with a ``configure`` script, Spack won't realize it uses
Autotools. You can overwrite the old package with ``--force`` and specify
a name with ``--name`` or a build system template to use with ``--template``:
.. code-block:: console .. code-block:: console
$ spack edit foo $ spack create --name gmp https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2
==> Error: No package 'foo'. Use spack create, or supply -f/--force to edit a new file. $ spack create --force --template autotools https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2
.. _spack-edit-f:
^^^^^^^^^^^^^^^^^^^^^^
``spack edit --force``
^^^^^^^^^^^^^^^^^^^^^^
``spack edit --force`` can be used to create a new, minimal boilerplate .. note::
package:
.. code-block:: console If you are creating a package that uses the Autotools build system
but does not come with a ``configure`` script, you'll need to add an
``autoreconf`` method to your package that explains how to generate
the ``configure`` script. You may also need the following dependencies:
$ spack edit --force foo .. code-block:: python
Unlike ``spack create``, which infers names and versions, and which depends_on('autoconf', type='build')
actually downloads the tarball and checksums it for you, ``spack edit depends_on('automake', type='build')
--force`` has no such fanciness. It will substitute dummy values for you depends_on('libtool', type='build')
to fill in yourself: depends_on('m4', type='build')
.. code-block:: python A complete list of available build system templates can be found by running
:linenos: ``spack create --help``.
from spack import * .. _cmd-spack-edit:
class Foo(Package): ^^^^^^^^^^^^^^
"""Description""" ``spack edit``
^^^^^^^^^^^^^^
homepage = "http://www.example.com" One of the easiest ways to learn how to write packages is to look at
url = "http://www.example.com/foo-1.0.tar.gz" existing ones. You can edit a package file by name with the ``spack
edit`` command:
version('1.0', '0123456789abcdef0123456789abcdef') .. code-block:: console
def install(self, spec, prefix): $ spack edit gmp
configure("--prefix=%s" % prefix)
make()
make("install")
This is useful when ``spack create`` cannot figure out the name and So, if you used ``spack create`` to create a package, then saved and
version of your package from the archive URL. closed the resulting file, you can get back to it with ``spack edit``.
The ``gmp`` package actually lives in
``$SPACK_ROOT/var/spack/repos/builtin/packages/gmp/package.py``,
but ``spack edit`` provides a much simpler shortcut and saves you the
trouble of typing the full path.
---------------------------- ----------------------------
Naming & directory structure Naming & directory structure
---------------------------- ----------------------------
This section describes how packages need to be named, and where they This section describes how packages need to be named, and where they
live in Spack's directory structure. In general, :ref:`cmd-spack-create` and live in Spack's directory structure. In general, :ref:`cmd-spack-create`
:ref:`cmd-spack-edit` handle creating package files for you, so you can skip handles creating package files for you, so you can skip most of the
most of the details here. details here.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
``var/spack/repos/builtin/packages`` ``var/spack/repos/builtin/packages``
...@@ -308,10 +293,9 @@ directories or files (like patches) that it needs to build. ...@@ -308,10 +293,9 @@ directories or files (like patches) that it needs to build.
Package Names Package Names
^^^^^^^^^^^^^ ^^^^^^^^^^^^^
Packages are named after the directory containing ``package.py``. It is Packages are named after the directory containing ``package.py``. So,
preferred, but not required, that the directory, and thus the package name, are ``libelf``'s ``package.py`` lives in a directory called ``libelf``.
lower case. So, ``libelf``'s ``package.py`` lives in a directory called The ``package.py`` file defines a class called ``Libelf``, which
``libelf``. The ``package.py`` file defines a class called ``Libelf``, which
extends Spack's ``Package`` class. For example, here is extends Spack's ``Package`` class. For example, here is
``$SPACK_ROOT/var/spack/repos/builtin/packages/libelf/package.py``: ``$SPACK_ROOT/var/spack/repos/builtin/packages/libelf/package.py``:
...@@ -336,21 +320,22 @@ these: ...@@ -336,21 +320,22 @@ these:
.. code-block:: console .. code-block:: console
$ spack install libelf $ spack info libelf
$ spack versions libelf
$ spack install libelf@0.8.13 $ spack install libelf@0.8.13
Spack sees the package name in the spec and looks for Spack sees the package name in the spec and looks for
``libelf/package.py`` in ``var/spack/repos/builtin/packages``. Likewise, if you say ``libelf/package.py`` in ``var/spack/repos/builtin/packages``.
``spack install py-numpy``, then Spack looks for Likewise, if you run ``spack install py-numpy``, Spack looks for
``py-numpy/package.py``. ``py-numpy/package.py``.
Spack uses the directory name as the package name in order to give Spack uses the directory name as the package name in order to give
packagers more freedom in naming their packages. Package names can packagers more freedom in naming their packages. Package names can
contain letters, numbers, dashes, and underscores. Using a Python contain letters, numbers, and dashes. Using a Python identifier
identifier (e.g., a class name or a module name) would make it (e.g., a class name or a module name) would make it difficult to
difficult to support these options. So, you can name a package support these options. So, you can name a package ``3proxy`` or
``3proxy`` or ``_foo`` and Spack won't care. It just needs to see ``foo-bar`` and Spack won't care. It just needs to see that name
that name in the package spec. in the packages directory.
^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
Package class names Package class names
...@@ -359,16 +344,14 @@ Package class names ...@@ -359,16 +344,14 @@ Package class names
Spack loads ``package.py`` files dynamically, and it needs to find a Spack loads ``package.py`` files dynamically, and it needs to find a
special class name in the file for the load to succeed. The **class special class name in the file for the load to succeed. The **class
name** (``Libelf`` in our example) is formed by converting words name** (``Libelf`` in our example) is formed by converting words
separated by `-` or ``_`` in the file name to camel case. If the name separated by ``-`` in the file name to CamelCase. If the name
starts with a number, we prefix the class name with ``_``. Here are starts with a number, we prefix the class name with ``_``. Here are
some examples: some examples:
================= ================= ================= =================
Module Name Class Name Module Name Class Name
================= ================= ================= =================
``foo_bar`` ``FooBar`` ``foo-bar`` ``FooBar``
``docbook-xml`` ``DocbookXml``
``FooBar`` ``Foobar``
``3proxy`` ``_3proxy`` ``3proxy`` ``_3proxy``
================= ================= ================= =================
...@@ -2719,7 +2702,7 @@ running: ...@@ -2719,7 +2702,7 @@ running:
from spack import * from spack import *
This is already part of the boilerplate for packages created with This is already part of the boilerplate for packages created with
``spack create`` or ``spack edit``. ``spack create``.
^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
Filtering functions Filtering functions
......
lib/spack/spack/cmd/checksum.py
+ 90
− 48
View file @ 1f49493f
...@@ -22,6 +22,8 @@ ...@@ -22,6 +22,8 @@
# License along with this program; if not, write to the Free Software # License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
############################################################################## ##############################################################################
from __future__ import print_function
import argparse import argparse
import hashlib import hashlib
...@@ -30,6 +32,7 @@ ...@@ -30,6 +32,7 @@
import spack.cmd import spack.cmd
import spack.util.crypto import spack.util.crypto
from spack.stage import Stage, FailedDownloadError from spack.stage import Stage, FailedDownloadError
from spack.util.naming import *
from spack.version import * from spack.version import *
description = "Checksum available versions of a package." description = "Checksum available versions of a package."
...@@ -37,86 +40,125 @@ ...@@ -37,86 +40,125 @@
def setup_parser(subparser): def setup_parser(subparser):
subparser.add_argument( subparser.add_argument(
'package', metavar='PACKAGE', help='Package to list versions for') 'package',
help='Package to checksum versions for')
subparser.add_argument( subparser.add_argument(
'--keep-stage', action='store_true', dest='keep_stage', '--keep-stage', action='store_true',
help="Don't clean up staging area when command completes.") help="Don't clean up staging area when command completes.")
subparser.add_argument( subparser.add_argument(
'versions', nargs=argparse.REMAINDER, 'versions', nargs=argparse.REMAINDER,
help='Versions to generate checksums for') help='Versions to generate checksums for')
def get_checksums(versions, urls, **kwargs): def get_checksums(url_dict, name, **kwargs):
# Allow commands like create() to do some analysis on the first """Fetches and checksums archives from URLs.
# archive after it is downloaded.
This function is called by both ``spack checksum`` and ``spack create``.
The ``first_stage_function`` kwarg allows ``spack create`` to determine
things like the build system of the archive.
:param dict url_dict: A dictionary of the form: version -> URL
:param str name: The name of the package
:param callable first_stage_function: Function to run on first staging area
:param bool keep_stage: Don't clean up staging area when command completes
:returns: A multi-line string containing versions and corresponding hashes
:rtype: str
"""
first_stage_function = kwargs.get('first_stage_function', None) first_stage_function = kwargs.get('first_stage_function', None)
keep_stage = kwargs.get('keep_stage', False) keep_stage = kwargs.get('keep_stage', False)
sorted_versions = sorted(url_dict.keys(), reverse=True)
# Find length of longest string in the list for padding
max_len = max(len(str(v)) for v in sorted_versions)
num_ver = len(sorted_versions)
tty.msg("Found {0} version{1} of {2}:".format(
num_ver, '' if num_ver == 1 else 's', name),
"",
*spack.cmd.elide_list(
["{0:{1}} {2}".format(v, max_len, url_dict[v])
for v in sorted_versions]))
print()
archives_to_fetch = tty.get_number(
"How many would you like to checksum?", default=1, abort='q')
if not archives_to_fetch:
tty.die("Aborted.")
versions = sorted_versions[:archives_to_fetch]
urls = [url_dict[v] for v in versions]
tty.msg("Downloading...") tty.msg("Downloading...")
hashes = [] version_hashes = []
i = 0 i = 0
for url, version in zip(urls, versions): for url, version in zip(urls, versions):
try: try:
with Stage(url, keep=keep_stage) as stage: with Stage(url, keep=keep_stage) as stage:
# Fetch the archive
stage.fetch() stage.fetch()
if i == 0 and first_stage_function: if i == 0 and first_stage_function:
# Only run first_stage_function the first time,
# no need to run it every time
first_stage_function(stage, url) first_stage_function(stage, url)
hashes.append((version, spack.util.crypto.checksum( # Checksum the archive and add it to the list
version_hashes.append((version, spack.util.crypto.checksum(
hashlib.md5, stage.archive_file))) hashlib.md5, stage.archive_file)))
i += 1 i += 1
except FailedDownloadError as e: except FailedDownloadError:
tty.msg("Failed to fetch %s" % url) tty.msg("Failed to fetch {0}".format(url))
except Exception as e: except Exception as e:
tty.msg('Something failed on %s, skipping.\n (%s)' % (url, e)) tty.msg("Something failed on {0}, skipping.".format(url),
" ({0})".format(e))
return hashes if not version_hashes:
tty.die("Could not fetch any versions for {0}".format(name))
# Find length of longest string in the list for padding
max_len = max(len(str(v)) for v, h in version_hashes)
# Generate the version directives to put in a package.py
version_lines = "\n".join([
" version('{0}', {1}'{2}')".format(
v, ' ' * (max_len - len(str(v))), h) for v, h in version_hashes
])
num_hash = len(version_hashes)
tty.msg("Checksummed {0} version{1} of {2}".format(
num_hash, '' if num_hash == 1 else 's', name))
return version_lines
def checksum(parser, args): def checksum(parser, args):
# get the package we're going to generate checksums for # Make sure the user provided a package and not a URL
if not valid_fully_qualified_module_name(args.package):
tty.die("`spack checksum` accepts package names, not URLs. "
"Use `spack md5 <url>` instead.")
# Get the package we're going to generate checksums for
pkg = spack.repo.get(args.package) pkg = spack.repo.get(args.package)
# If the user asked for specific versions, use those.
if args.versions: if args.versions:
versions = {} # If the user asked for specific versions, use those
url_dict = {}
for version in args.versions: for version in args.versions:
version = ver(version) version = ver(version)
if not isinstance(version, Version): if not isinstance(version, Version):
tty.die("Cannot generate checksums for version lists or " + tty.die("Cannot generate checksums for version lists or "
"version ranges. Use unambiguous versions.") "version ranges. Use unambiguous versions.")
versions[version] = pkg.url_for_version(version) url_dict[version] = pkg.url_for_version(version)
else: else:
versions = pkg.fetch_remote_versions() # Otherwise, see what versions we can find online
if not versions: url_dict = pkg.fetch_remote_versions()
tty.die("Could not fetch any versions for %s" % pkg.name) if not url_dict:
tty.die("Could not find any versions for {0}".format(pkg.name))
sorted_versions = sorted(versions, reverse=True)
# Find length of longest string in the list for padding
maxlen = max(len(str(v)) for v in versions)
tty.msg("Found %s versions of %s" % (len(versions), pkg.name), version_lines = get_checksums(
*spack.cmd.elide_list( url_dict, pkg.name, keep_stage=args.keep_stage)
["{0:{1}} {2}".format(v, maxlen, versions[v])
for v in sorted_versions]))
print
archives_to_fetch = tty.get_number(
"How many would you like to checksum?", default=5, abort='q')
if not archives_to_fetch:
tty.msg("Aborted.")
return
version_hashes = get_checksums(
sorted_versions[:archives_to_fetch],
[versions[v] for v in sorted_versions[:archives_to_fetch]],
keep_stage=args.keep_stage)
if not version_hashes:
tty.die("Could not fetch any versions for %s" % pkg.name)
version_lines = [ print()
" version('%s', '%s')" % (v, h) for v, h in version_hashes print(version_lines)
]
tty.msg("Checksummed new versions of %s:" % pkg.name, *version_lines)
lib/spack/spack/cmd/create.py
+ 262
− 176
View file @ 1f49493f
This diff is collapsed.
lib/spack/spack/cmd/diy.py
+ 2
− 10
View file @ 1f49493f
...@@ -31,7 +31,6 @@ ...@@ -31,7 +31,6 @@
import spack import spack
import spack.cmd import spack.cmd
import spack.cmd.common.arguments as arguments import spack.cmd.common.arguments as arguments
from spack.cmd.edit import edit_package
from spack.stage import DIYStage from spack.stage import DIYStage
description = "Do-It-Yourself: build from an existing source directory." description = "Do-It-Yourself: build from an existing source directory."
...@@ -68,15 +67,8 @@ def diy(self, args): ...@@ -68,15 +67,8 @@ def diy(self, args):
spec = specs[0] spec = specs[0]
if not spack.repo.exists(spec.name): if not spack.repo.exists(spec.name):
tty.warn("No such package: %s" % spec.name) tty.die("No package for '{0}' was found.".format(spec.name),
create = tty.get_yes_or_no("Create this package?", default=False) " Use `spack create` to create a new package")
if not create:
tty.msg("Exiting without creating.")
sys.exit(1)
else:
tty.msg("Running 'spack edit -f %s'" % spec.name)
edit_package(spec.name, spack.repo.first_repo(), None, True)
return
if not spec.versions.concrete: if not spec.versions.concrete:
tty.die( tty.die(
......
lib/spack/spack/cmd/edit.py
+ 24
− 42
View file @ 1f49493f
...@@ -23,39 +23,26 @@ ...@@ -23,39 +23,26 @@
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
############################################################################## ##############################################################################
import os import os
import string
import llnl.util.tty as tty import llnl.util.tty as tty
from llnl.util.filesystem import mkdirp, join_path from llnl.util.filesystem import join_path
import spack import spack
import spack.cmd import spack.cmd
from spack.spec import Spec from spack.spec import Spec
from spack.repository import Repo from spack.repository import Repo
from spack.util.naming import mod_to_class
description = "Open package files in $EDITOR" description = "Open package files in $EDITOR"
# When -f is supplied, we'll create a very minimal skeleton.
package_template = string.Template("""\
from spack import *
class ${class_name}(Package): def edit_package(name, repo_path, namespace):
""\"Description""\" """Opens the requested package file in your favorite $EDITOR.
homepage = "http://www.example.com" :param str name: The name of the package
url = "http://www.example.com/${name}-1.0.tar.gz" :param str repo_path: The path to the repository containing this package
:param str namespace: A valid namespace registered with Spack
version('1.0', '0123456789abcdef0123456789abcdef') """
# Find the location of the package
def install(self, spec, prefix):
configure("--prefix=%s" % prefix)
make()
make("install")
""")
def edit_package(name, repo_path, namespace, force=False):
if repo_path: if repo_path:
repo = Repo(repo_path) repo = Repo(repo_path)
elif namespace: elif namespace:
...@@ -67,37 +54,29 @@ def edit_package(name, repo_path, namespace, force=False): ...@@ -67,37 +54,29 @@ def edit_package(name, repo_path, namespace, force=False):
spec = Spec(name) spec = Spec(name)
if os.path.exists(path): if os.path.exists(path):
if not os.path.isfile(path): if not os.path.isfile(path):
tty.die("Something's wrong. '%s' is not a file!" % path) tty.die("Something is wrong. '{0}' is not a file!".format(path))
if not os.access(path, os.R_OK | os.W_OK): if not os.access(path, os.R_OK | os.W_OK):
tty.die("Insufficient permissions on '%s'!" % path) tty.die("Insufficient permissions on '%s'!" % path)
elif not force:
tty.die("No package '%s'. Use spack create, or supply -f/--force "
"to edit a new file." % spec.name)
else: else:
mkdirp(os.path.dirname(path)) tty.die("No package for '{0}' was found.".format(spec.name),
with open(path, "w") as pkg_file: " Use `spack create` to create a new package")
pkg_file.write(
package_template.substitute(
name=spec.name, class_name=mod_to_class(spec.name)))
spack.editor(path) spack.editor(path)
def setup_parser(subparser): def setup_parser(subparser):
subparser.add_argument(
'-f', '--force', dest='force', action='store_true',
help="Open a new file in $EDITOR even if package doesn't exist.")
excl_args = subparser.add_mutually_exclusive_group() excl_args = subparser.add_mutually_exclusive_group()
# Various filetypes you can edit directly from the cmd line. # Various types of Spack files that can be edited
# Edits package files by default
excl_args.add_argument( excl_args.add_argument(
'-c', '--command', dest='path', action='store_const', '-c', '--command', dest='path', action='store_const',
const=spack.cmd.command_path, const=spack.cmd.command_path,
help="Edit the command with the supplied name.") help="Edit the command with the supplied name.")
excl_args.add_argument( excl_args.add_argument(
'-t', '--test', dest='path', action='store_const', '-t', '--test', dest='path', action='store_const',
const=spack.test_path, help="Edit the test with the supplied name.") const=spack.test_path,
help="Edit the test with the supplied name.")
excl_args.add_argument( excl_args.add_argument(
'-m', '--module', dest='path', action='store_const', '-m', '--module', dest='path', action='store_const',
const=spack.module_path, const=spack.module_path,
...@@ -112,23 +91,26 @@ def setup_parser(subparser): ...@@ -112,23 +91,26 @@ def setup_parser(subparser):
help="Namespace of package to edit.") help="Namespace of package to edit.")
subparser.add_argument( subparser.add_argument(
'name', nargs='?', default=None, help="name of package to edit") 'name', nargs='?', default=None,
help="name of package to edit")
def edit(parser, args): def edit(parser, args):
name = args.name name = args.name
# By default, edit package files
path = spack.packages_path path = spack.packages_path
# If `--command`, `--test`, or `--module` is chosen, edit those instead
if args.path: if args.path:
path = args.path path = args.path
if name: if name:
path = join_path(path, name + ".py") path = join_path(path, name + ".py")
if not args.force and not os.path.exists(path): if not os.path.exists(path):
tty.die("No command named '%s'." % name) tty.die("No command for '{0}' was found.".format(name))
spack.editor(path) spack.editor(path)
elif name: elif name:
edit_package(name, args.repo, args.namespace, args.force) edit_package(name, args.repo, args.namespace)
else: else:
# By default open the directory where packages or commands live. # By default open the directory where packages live
spack.editor(path) spack.editor(path)
lib/spack/spack/cmd/setup.py
+ 2
− 11
View file @ 1f49493f
...@@ -36,7 +36,6 @@ ...@@ -36,7 +36,6 @@
import spack.cmd.common.arguments as arguments import spack.cmd.common.arguments as arguments
from llnl.util.filesystem import set_executable from llnl.util.filesystem import set_executable
from spack import which from spack import which
from spack.cmd.edit import edit_package
from spack.stage import DIYStage from spack.stage import DIYStage
description = "Create a configuration script and module, but don't build." description = "Create a configuration script and module, but don't build."
...@@ -134,16 +133,8 @@ def setup(self, args): ...@@ -134,16 +133,8 @@ def setup(self, args):
with spack.store.db.write_transaction(): with spack.store.db.write_transaction():
spec = specs[0] spec = specs[0]
if not spack.repo.exists(spec.name): if not spack.repo.exists(spec.name):
tty.warn("No such package: %s" % spec.name) tty.die("No package for '{0}' was found.".format(spec.name),
create = tty.get_yes_or_no("Create this package?", default=False) " Use `spack create` to create a new package")
if not create:
tty.msg("Exiting without creating.")
sys.exit(1)
else:
tty.msg("Running 'spack edit -f %s'" % spec.name)
edit_package(spec.name, spack.repo.first_repo(), None, True)
return
if not spec.versions.concrete: if not spec.versions.concrete:
tty.die( tty.die(
"spack setup spec must have a single, concrete version. " "spack setup spec must have a single, concrete version. "
......
lib/spack/spack/test/build_system_guess.py
+ 6
− 5
View file @ 1f49493f
...@@ -32,12 +32,13 @@ ...@@ -32,12 +32,13 @@
@pytest.fixture( @pytest.fixture(
scope='function', scope='function',
params=[ params=[
('configure', 'autotools'), ('configure', 'autotools'),
('CMakeLists.txt', 'cmake'), ('CMakeLists.txt', 'cmake'),
('SConstruct', 'scons'), ('SConstruct', 'scons'),
('setup.py', 'python'), ('setup.py', 'python'),
('NAMESPACE', 'r'), ('NAMESPACE', 'r'),
('foobar', 'unknown') ('WORKSPACE', 'bazel'),
('foobar', 'generic')
] ]
) )
def url_and_build_system(request, tmpdir): def url_and_build_system(request, tmpdir):
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment