[SciPy-Dev] CUTEst in Scipy

[Antonio Ribeiro]

Hello,

I am developing my google of summer proposal about constrained optimisation
in Scipy
and it will be very important to have a good set of benchmarks.

There is a great library with diverse optimisation benchmarks called CUTEst
<https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki/>. It is under LGPL
2.1 license.

This CUTEst library include a huge amount of problem sets
and is often used in optimisation papers. Furthermore, many of the available
optimisation software provide some interface to it. I am interested in
using problems from this set in my project and I want to ask how
should I proceed?

1) Select a subset of tests from the CUTEst library and implement them in
native python under scipy.

2) Include some interface to CUTEst in scipy. By what I understand LGPL
license is
more restrictive than BSD-3 used by Scipy. In this case, could we ask for
permission?

3) There is an interface for CUTEst implemented in the library pyOpus
(under GPL3 license)
<http://fides.fe.uni-lj.si/pyopus/download1.html>. Since this is a library
external to Scipy (with an incompatible license) how should I proceed in
this case: can I make my benchmarks dependent on it? Can we ask permission
for include this interface in Scipy?

Any suggestions on how to proceed and what to include in my proposal is
very welcome.

AntÃŽnio

[Nikolay Mayorov]

Hi!



I heard about CUTEst, but it looked quite complex so I decided not to deal with it. To your options:



1) It might be the most practical option, considering you'll have other (main) parts of work to do. Also it's not clear whether to include test problems in scipy or store them somewhere else. My opinion is that having a collection of problems right in scipy.optimize is good. I don't exactly remember why we decided not to include test problems for least_squares, I guess because it required many considerations. In theory there are ASV benchmarks in scipy, but they don't feel adequate for benchmarking optimization problems. So developing benchmarking infrastructure in scipy.optimize might be a good side project, my concern is that it will distract you from the main path greatly.



To sum up: the simplest way is to pick some problems, write your own benchmark suite and store it somewhere else. This way you will be able to focus on the algorithms. Other options are possible if you have a really good idea how to do them and ready to devote time to it.



2) My understanding is that you don't need any permissions as you won't use CUTEst code (someone could correct me). As I see it: your add an interface into scipy, a user install CUTEst and use this interface to work with CUTEst. It doesn't seem very practical, because installing CUTEst looks like a pain. Do you agree or I am mistaken?



3) I guess you can ask an author to reuse his code in scipy to implement an interface to CUTEst.



Generally about 2 and 3: it might be a good idea to use CUTEst for your internal benchmarking during the development and optionally you can add an interface to CUTEst into scipy. But I think it will be hardly ever used after that.



To sum the idea: adding CUTEst dependency for scipy.optimize benchmarking seems unpractical. It would be useful if you will actually try to work with CUTEst, maybe it will turn out to be not that difficult. In such case many of my points are not valid.



I hope you'll be able to make some sense from my rambling.


Nikolay.




---- On Mon, 20 Mar 2017 23:24:14 +0500 Antonio Ribeiro &lt;***@gmail.com&gt; wrote ----




Hello,



I am developing my google of summer proposal about constrained optimisation in Scipy

and it will be very important to have a good set of benchmarks.



There is a great library with diverse optimisation benchmarks called CUTEst &lt;https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki/&gt;. It is under LGPL 2.1 license.



This CUTEst library include a huge amount of problem sets

and is often used in optimisation papers. Furthermore, many of the available

optimisation software provide some interface to it. I am interested in

using problems from this set in my project and I want to ask how

should I proceed?




1) Select a subset of tests from the CUTEst library and implement them in native python under scipy.



2) Include some interface to CUTEst in scipy. By what I understand LGPL license is

more restrictive than BSD-3 used by Scipy. In this case, could we ask for permission?



3) There is an interface for CUTEst implemented in the library pyOpus (under GPL3 license)

&lt;http://fides.fe.uni-lj.si/pyopus/download1.html&gt;. Since this is a library external to Scipy (with an incompatible license) how should I proceed in this case: can I make my benchmarks dependent on it? Can we ask permission for include this interface in Scipy?





Any suggestions on how to proceed and what to include in my proposal is very welcome.



AntÃŽnio



_______________________________________________

SciPy-Dev mailing list

SciPy-***@scipy.org

https://mail.scipy.org/mailman/listinfo/scipy-dev

[Andrew Nelson]

There are some least squares problems from the NIST test suite in the
benchmarks/benchmarks/go_benchmark_functions, but they're couched in terms
of a simple scalar (chi2) minimisation problem, and they are all bounded
problems.

Post by Nikolay Mayorov
Hi!
I heard about CUTEst, but it looked quite complex so I decided not to deal
1) It might be the most practical option, considering you'll have other
(main) parts of work to do. Also it's not clear whether to include test
problems in scipy or store them somewhere else. My opinion is that having a
collection of problems right in scipy.optimize is good. I don't exactly
remember why we decided not to include test problems for least_squares, I
guess because it required many considerations. In theory there are ASV
benchmarks in scipy, but they don't feel adequate for benchmarking
optimization problems. So developing benchmarking infrastructure in
scipy.optimize might be a good side project, my concern is that it will
distract you from the main path greatly.
To sum up: the simplest way is to pick some problems, write your own
benchmark suite and store it somewhere else. This way you will be able to
focus on the algorithms. Other options are possible if you have a really
good idea how to do them and ready to devote time to it.
2) My understanding is that you don't need any permissions as you won't
use CUTEst code (someone could correct me). As I see it: your add an
interface into scipy, a user install CUTEst and use this interface to work
with CUTEst. It doesn't seem very practical, because installing CUTEst
looks like a pain. Do you agree or I am mistaken?
3) I guess you can ask an author to reuse his code in scipy to implement
an interface to CUTEst.
Generally about 2 and 3: it might be a good idea to use CUTEst for your
internal benchmarking during the development and optionally you can add an
interface to CUTEst into scipy. But I think it will be hardly ever used
after that.
To sum the idea: adding CUTEst dependency for scipy.optimize benchmarking
seems unpractical. It would be useful if you will actually try to work with
CUTEst, maybe it will turn out to be not that difficult. In such case many
of my points are not valid.
I hope you'll be able to make some sense from my rambling.
Nikolay.
---- On Mon, 20 Mar 2017 23:24:14 +0500 *Antonio Ribeiro
Hello,
I am developing my google of summer proposal about constrained
optimisation in Scipy
and it will be very important to have a good set of benchmarks.
There is a great library with diverse optimisation benchmarks called
CUTEst <https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki/>. It is
under LGPL 2.1 license.
This CUTEst library include a huge amount of problem sets
and is often used in optimisation papers. Furthermore, many of the available
optimisation software provide some interface to it. I am interested in
using problems from this set in my project and I want to ask how
should I proceed?
1) Select a subset of tests from the CUTEst library and implement them in
native python under scipy.
2) Include some interface to CUTEst in scipy. By what I understand LGPL license is
more restrictive than BSD-3 used by Scipy. In this case, could we ask for permission?
3) There is an interface for CUTEst implemented in the library pyOpus (under GPL3 license)
<http://fides.fe.uni-lj.si/pyopus/download1.html>. Since this is a
library external to Scipy (with an incompatible license) how should I
proceed in this case: can I make my benchmarks dependent on it? Can we ask
permission for include this interface in Scipy?
Any suggestions on how to proceed and what to include in my proposal is very welcome.
AntÃŽnio
_______________________________________________
SciPy-Dev mailing list
https://mail.scipy.org/mailman/listinfo/scipy-dev
_______________________________________________
SciPy-Dev mailing list
https://mail.scipy.org/mailman/listinfo/scipy-dev

[Antonio Ribeiro]

Thank you for the insightful answer Nikolay! I agree with
you that include CUTEst dependency to Scipy could
be troublesome, and furthermore it could be very painful
to make this interface work well for more than one
operational systems... The solution of implementing
and store the benchmark somewhere else is not
ideal but seems a good option because it make things
simpler.

About the use of NIST test suite: Thank you for the suggestion
Andrew, but I don't think it is a good test suit for the methods
I intend to implement.

Send SciPy-Dev mailing list submissions to
To subscribe or unsubscribe via the World Wide Web, visit
https://mail.scipy.org/mailman/listinfo/scipy-dev
or, via email, send a message with subject or body 'help' to
You can reach the person managing the list at
When replying, please edit your Subject line so it is more specific
than "Re: Contents of SciPy-Dev digest..."
1. CUTEst in Scipy (Antonio Ribeiro)
2. Re: CUTEst in Scipy (Nikolay Mayorov)
3. Re: CUTEst in Scipy (Andrew Nelson)
----------------------------------------------------------------------
Message: 1
Date: Mon, 20 Mar 2017 15:24:14 -0300
Subject: [SciPy-Dev] CUTEst in Scipy
gmail.com>
Content-Type: text/plain; charset="utf-8"
Hello,
I am developing my google of summer proposal about constrained optimisation
in Scipy
and it will be very important to have a good set of benchmarks.
There is a great library with diverse optimisation benchmarks called CUTEst
<https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki/>. It is under LGPL
2.1 license.
This CUTEst library include a huge amount of problem sets
and is often used in optimisation papers. Furthermore, many of the
available
optimisation software provide some interface to it. I am interested in
using problems from this set in my project and I want to ask how
should I proceed?
1) Select a subset of tests from the CUTEst library and implement them in
native python under scipy.
2) Include some interface to CUTEst in scipy. By what I understand LGPL
license is
more restrictive than BSD-3 used by Scipy. In this case, could we ask for
permission?
3) There is an interface for CUTEst implemented in the library pyOpus
(under GPL3 license)
<http://fides.fe.uni-lj.si/pyopus/download1.html>. Since this is a
library
external to Scipy (with an incompatible license) how should I proceed in
this case: can I make my benchmarks dependent on it? Can we ask permission
for include this interface in Scipy?
Any suggestions on how to proceed and what to include in my proposal is
very welcome.
Ant?nio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.scipy.org/pipermail/scipy-dev/
attachments/20170320/f2417724/attachment-0001.html>
------------------------------
Message: 2
Date: Tue, 21 Mar 2017 00:26:55 +0500
Subject: Re: [SciPy-Dev] CUTEst in Scipy
Content-Type: text/plain; charset="utf-8"
Hi!
I heard about CUTEst, but it looked quite complex so I decided not to deal
1) It might be the most practical option, considering you'll have other
(main) parts of work to do. Also it's not clear whether to include test
problems in scipy or store them somewhere else. My opinion is that having a
collection of problems right in scipy.optimize is good. I don't exactly
remember why we decided not to include test problems for least_squares, I
guess because it required many considerations. In theory there are ASV
benchmarks in scipy, but they don't feel adequate for benchmarking
optimization problems. So developing benchmarking infrastructure in
scipy.optimize might be a good side project, my concern is that it will
distract you from the main path greatly.
To sum up: the simplest way is to pick some problems, write your own
benchmark suite and store it somewhere else. This way you will be able to
focus on the algorithms. Other options are possible if you have a really
good idea how to do them and ready to devote time to it.
2) My understanding is that you don't need any permissions as you won't
use CUTEst code (someone could correct me). As I see it: your add an
interface into scipy, a user install CUTEst and use this interface to work
with CUTEst. It doesn't seem very practical, because installing CUTEst
looks like a pain. Do you agree or I am mistaken?
3) I guess you can ask an author to reuse his code in scipy to implement
an interface to CUTEst.
Generally about 2 and 3: it might be a good idea to use CUTEst for your
internal benchmarking during the development and optionally you can add an
interface to CUTEst into scipy. But I think it will be hardly ever used
after that.
To sum the idea: adding CUTEst dependency for scipy.optimize benchmarking
seems unpractical. It would be useful if you will actually try to work with
CUTEst, maybe it will turn out to be not that difficult. In such case many
of my points are not valid.
I hope you'll be able to make some sense from my rambling.
Nikolay.
---- On Mon, 20 Mar 2017 23:24:14 +0500 Antonio Ribeiro &
Hello,
I am developing my google of summer proposal about constrained
optimisation in Scipy
and it will be very important to have a good set of benchmarks.
There is a great library with diverse optimisation benchmarks called
is under LGPL 2.1 license.
This CUTEst library include a huge amount of problem sets
and is often used in optimisation papers. Furthermore, many of the
available
optimisation software provide some interface to it. I am interested in
using problems from this set in my project and I want to ask how
should I proceed?
1) Select a subset of tests from the CUTEst library and implement them in
native python under scipy.
2) Include some interface to CUTEst in scipy. By what I understand LGPL
license is
more restrictive than BSD-3 used by Scipy. In this case, could we ask for
permission?
3) There is an interface for CUTEst implemented in the library pyOpus
(under GPL3 license)
library external to Scipy (with an incompatible license) how should I
proceed in this case: can I make my benchmarks dependent on it? Can we ask
permission for include this interface in Scipy?
Any suggestions on how to proceed and what to include in my proposal is
very welcome.
Ant?nio
_______________________________________________
SciPy-Dev mailing list
https://mail.scipy.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.scipy.org/pipermail/scipy-dev/
attachments/20170321/751cb63f/attachment-0001.html>
------------------------------
Message: 3
Date: Tue, 21 Mar 2017 09:21:59 +1100
Subject: Re: [SciPy-Dev] CUTEst in Scipy
gmail.com>
Content-Type: text/plain; charset="utf-8"
There are some least squares problems from the NIST test suite in the
benchmarks/benchmarks/go_benchmark_functions, but they're couched in terms
of a simple scalar (chi2) minimisation problem, and they are all bounded
problems.

Post by Nikolay Mayorov
Hi!
I heard about CUTEst, but it looked quite complex so I decided not to

deal

Post by Nikolay Mayorov
1) It might be the most practical option, considering you'll have other
(main) parts of work to do. Also it's not clear whether to include test
problems in scipy or store them somewhere else. My opinion is that

having a

Post by Nikolay Mayorov
collection of problems right in scipy.optimize is good. I don't exactly
remember why we decided not to include test problems for least_squares, I
guess because it required many considerations. In theory there are ASV
benchmarks in scipy, but they don't feel adequate for benchmarking
optimization problems. So developing benchmarking infrastructure in
scipy.optimize might be a good side project, my concern is that it will
distract you from the main path greatly.
To sum up: the simplest way is to pick some problems, write your own
benchmark suite and store it somewhere else. This way you will be able to
focus on the algorithms. Other options are possible if you have a really
good idea how to do them and ready to devote time to it.
2) My understanding is that you don't need any permissions as you won't
use CUTEst code (someone could correct me). As I see it: your add an
interface into scipy, a user install CUTEst and use this interface to

work

Post by Nikolay Mayorov
with CUTEst. It doesn't seem very practical, because installing CUTEst
looks like a pain. Do you agree or I am mistaken?
3) I guess you can ask an author to reuse his code in scipy to implement
an interface to CUTEst.
Generally about 2 and 3: it might be a good idea to use CUTEst for your
internal benchmarking during the development and optionally you can add

an

Post by Nikolay Mayorov
interface to CUTEst into scipy. But I think it will be hardly ever used
after that.
To sum the idea: adding CUTEst dependency for scipy.optimize benchmarking
seems unpractical. It would be useful if you will actually try to work

with

Post by Nikolay Mayorov
CUTEst, maybe it will turn out to be not that difficult. In such case

many

Post by Nikolay Mayorov
of my points are not valid.
I hope you'll be able to make some sense from my rambling.
Nikolay.
---- On Mon, 20 Mar 2017 23:24:14 +0500 *Antonio Ribeiro
Hello,
I am developing my google of summer proposal about constrained
optimisation in Scipy
and it will be very important to have a good set of benchmarks.
There is a great library with diverse optimisation benchmarks called
CUTEst <https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki/>. It is
under LGPL 2.1 license.
This CUTEst library include a huge amount of problem sets
and is often used in optimisation papers. Furthermore, many of the
available
optimisation software provide some interface to it. I am interested in
using problems from this set in my project and I want to ask how
should I proceed?
1) Select a subset of tests from the CUTEst library and implement them in
native python under scipy.
2) Include some interface to CUTEst in scipy. By what I understand LGPL
license is
more restrictive than BSD-3 used by Scipy. In this case, could we ask for
permission?
3) There is an interface for CUTEst implemented in the library pyOpus
(under GPL3 license)
<http://fides.fe.uni-lj.si/pyopus/download1.html>. Since this is a
library external to Scipy (with an incompatible license) how should I
proceed in this case: can I make my benchmarks dependent on it? Can we

ask

Post by Nikolay Mayorov
permission for include this interface in Scipy?
Any suggestions on how to proceed and what to include in my proposal is
very welcome.
Ant?nio
_______________________________________________
SciPy-Dev mailing list
https://mail.scipy.org/mailman/listinfo/scipy-dev
_______________________________________________
SciPy-Dev mailing list
https://mail.scipy.org/mailman/listinfo/scipy-dev

--
_____________________________________
Dr. Andrew Nelson
_____________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.scipy.org/pipermail/scipy-dev/
attachments/20170321/de9dde91/attachment.html>
------------------------------
Subject: Digest Footer
_______________________________________________
SciPy-Dev mailing list
https://mail.scipy.org/mailman/listinfo/scipy-dev
------------------------------
End of SciPy-Dev Digest, Vol 161, Issue 28
******************************************