Django’s (default) Permission Codenames

django, programming

Permission String

The documentation for Django authentication (e.g. https://docs.djangoproject.com/en/2.2/topics/auth/default/#permissions-and-authorization) talks about allowing code to restrict access to models through a vague notion of a “permission.”

Read further and you kinda get the idea that a “permission” in most practical applications is a string. Whether you’re using explicit checks like user.has_perm(permission) or decorators like @permission_required(permission) or class properties for the PermissionRequiredMixin like this (taken right out from the page):

from django.contrib.auth.mixins import PermissionRequiredMixin

class MyView(PermissionRequiredMixin, View):
    permission_required = 'polls.can_vote'

The problem with using strings is that the burden of getting things correct lies heavily on the documentation. A basic question, then, is that: given a model created in the app, what permission string should you use to work with the Authentication system to guard the model’s access?

Extremely Underrated Section

The short section on “Default permissions” (https://docs.djangoproject.com/en/2.2/topics/auth/default/#default-permissions) initially looks innocuous enough. However, it turns out that the section is the key to using permissions on models (custom permissions notwithstanding).

If you’re in a hurry to get permissions working on some models, you’d be forgiven for searching on the page and jump directly to the section on the “permission_required” decorator (and the section on PermissionRequiredMixin immediately after) where they show examples of using permissions of the form:

<model>.can_<action>

For example, 'polls.can_vote' is used throughout these sections to indicate the ability to “vote” on the “polls” model.

You may infer from that permissions like:

polls.can_add
polls.can_change
polls.can_delete
polls.can_view

because–oh–you skimmed some section earlier on the helper functions has_view_permission(), has_add_permission(), etc.

Well. Turns out these permission values will not work because the default permissions on models do not in fact follow this convention.

UPDATE: the Django 3.0 documentation, fortunately, fixed this by giving actual working values (e.g. “‘polls.add_choice‘”).

Convention on Default Permission Codenames

That brings us back to the section “Default permissions,” that small section of about 20 lines on a page of thousands.

Codename

Firstly, a permission (as can be seen in the table auth_permission) has an attribute “codename,” and that is what is used to communicate the permission being tested on.

Testing Against Codenames

All methods of working on permissions will operate on these codename values:

Model.has_perm(codename) and its friends from ModelAdmin: has_view_permission, has_add_permission, has_change_permission, and has_delete_permission

django.contrib.auth.decorators.permission_required decorator

PermissionRequiredMixin mixin class

Convention of Default Permission Codenames

Finally, the convention of the codename (as documented in that Default permissions section) is:

<app_label>.<action>_<model name>
  • The app_label is the name of the AppConfig for your app, most likely defined in apps.py of your app.
  • The action is one of ‘add’, ‘change’, ‘delete’, and (with Django version 2.1 or higher) ‘view.’
  • The model_name is the lower case form of your model name in the DB.

So if I had a Polls app with a model Posting,

polls/apps.py:

from django.apps import AppConfig
class PollsConfig(AppConfig):
    name = 'polls'

possible permission codenames are:

  • polls.view_posting
  • polls.add_posting
  • polls.change_posting
  • polls.delete_posting

When in Doubt, Check the DB

If you’re not sure, you can always check the auth_permission table in your DB:

select name, codename from auth_permission;

The name is what the Admin pages use to show the permission to assign/unassign. The codename is, of course, the permission string you would use in code.

More MVN Repository Notes

maven

More findings from working with MVN repository

When mvn release:perform is run, there are two steps that I didn’t pay attention to before. Of course, the building, signing, and uploading of JARs/POMs to the staging repository is expected from the various updates to the POMs seen from the previous post.

Here are the steps:

  • Creation of a staging repository
  • Building, signing, and uploading artifacts into the staging repository
  • Closing the staging repository
  • Releasing the staging repository

Looking at the logs of the release more closely, this can be seen:

...
    [INFO]  * Upload of locally staged artifacts finished.
    [INFO]  * Closing staging repository with ID "comtherealvan-1016".
         
    Waiting for operation to complete...

    ........                                                                                                              
    [INFO] Remote staged 1 repositories, finished with success.                    
    [INFO] Remote staging repositories are being released...                                                                                     
    Waiting for operation to complete...                                           
    .......                                                                                                               
    [INFO] Remote staging repositories released.             
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS                                     
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 01:39 min
...

Parent POMs

Yep. More notes about parent POMs that have <packaging>pom</packaging> and global props that submodules inherit:

Closing and releasing have to be done manually

A staging repository will be created. However, I have to log into https://oss.sonatype.org/ and look up the staging repository (I just search for my group ID “therealvan.com” to find my artifacts).

Then I have to select my staging repository and click “Close” and then, after it successfully closes, click “Release” to release the repository. There is also a “Drop” to remove the staging repository when it’s no longer needed (e.g. clean up after a successful release or aborting a release).

More information is expected

For parent POMs, I ran into errors when trying to close the staging repository. They have to do with missing:

  • Developer information
  • Licensing information
  • Project description
  • Project URL

These do not seem to be required (at least for now) when the closing and releasing are done as part of mvn release:perform. However, when closing the staging repository manually, these come up as errors.

Fortunately, they are easy to fix:

  • Developer information can be provided using the <developers> section in the POM.
  • Licensing information can be provided using the <licenses> section in the POM.
  • Project description and URL are just <description> and <url> tags for the top-level <project>.

 

 

Publishing a Java project into MVN Repository

Java, maven, programming

It turns out that this is quite an involved process with a lot of configuration and coordination w/ Sonatype. It took several days, upwards to a week or so, for me to get this working. Plan accordingly.

These notes are based on various articles out there, but updated to reflect learnings I had when going through the process.

Qualifications:

Here are some specifics for my project.

  • Project is hosted in GitHub as a public repo.
  • Project is a Maven project with a POM file at the root. It’s a multi-module project, but that shouldn’t change anything other than the fact that I publish the modules individually. There may be a way to publish all the modules simultaneously, but I haven’t explored that option.

Prerequisite:

  • Create an account with Sonatype. Start at https://issues.sonatype.org/secure/Dashboard.jspa and click “Sign up.”
  • Create an issue for the new project:
    • Project: “Community Support – Open Source Project Repository Hosting”
    • Type: “New Project”
    • Group Id: a domain I own (or something based on the project hosting like io.github.<user narme> or com.github.<user name>.
    • Project URL: GitHub URL to the project.
    • SCM URL: path to the repo (https://github.com/…/<project>.git)
    • Username(s): the Sonatype user name.

Configure project with Maven src and Javadoc plugins:

See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L106-L134

Create and publish an RSA key

  • Create a RSA key (e.g. using GnuPG’s “gpg –full-gen-key with 2048 bits)
  • Publish the key (e.g. “gpg –keyserver pool.sks-keyservers.net –send-key <my RSA key ID>“)

Configure Maven to talk to Sonatype servers:

Create/edit the file settings.xml under M2_HOME/config or M2_REPO/config:

Find the servers section and add:

<server>
  <id>ossrh</id>
  <username>my_sonatype_username</username>
  <password>my_sonatype_password</password>
</server>

Find the profiles section and add:

<profile>
  <id>ossrh</id>
  <activation>
    <activeByDefault>true</activeByDefault>
  </activation>
  <properties>
    <gpg.keyname>my RSA key ID</gpg.keyname>
    <gpg.passphrase>my RSA key's passphrase</gpg.passphrase>
  </properties>
</profile>

Use the same GPG key generated above.

Configure project for Maven DEPLOY Plugin:

Add the Maven Deploy Plugin. See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L64-L76

Add a distributionManagement section to the project POM. See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L137-L146

Add an scm section to the project POM. See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L147-L152

Configure Project for Nexus-Staging-Maven Plugin:

See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L95-L105

If using a different server ID than “ossrh,” keep in sync with the entries defined in the distributionManagement section and also the configuration in the Maven conf/settings.xml.

Configure project for Maven Release Plugin:

Add the Maven Release Plugin. See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L77-L94

Configure Project to Sign artifacts when releasing:

See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L154-L180

Preparation of Release:

Finally, to prepare for a project to be released,

  • Build the project once with mvn install
  • Fix all the issues that come up with unit tests and Javadoc.
  • Commit and push all the changes to GitHub.
  • Modify the project’s version to be a “-SNAPSHOT” release for the release I want to make. For example, if I want to release a 1.0.0 version, use the version “1.0.0-SNAPSHOT” for the project. However, none of the project’s dependencies can be to SNAPSHOT versions.
  • Commit the change. No need to push to GitHub.

Run mvn release:prepare

The process will ask some questions. The release will be “1.0.0” in this example. The next release is probably “1.0.1-SNAPSHOT” as suggested, but this can always be modified later as needed.

Release TO SONATYPE:

Run mvn release:perform

This takes a while, and it will actually push artifacts to Sonatype’s servers. Things can go wrong here that range from intermittent network errors to errors with the setup of the repo.

Artifacts can be verified by logging into https://oss.sonatype.org/ with the Sonatype account created earlier and searching for the released artifacts.

Issues may need to be filed with Sonatype if the repo is set up incompletely. Even when things work correctly, it may take some time for things to propagate through the various servers before the artifacts show up in mvnrepository.com.

Once this works the first time, subsequent releases are more stable.

 

Addendum:

Parent POMs

To release parent POMs w/o triggering releasing the modules under the parent, add:

 -N -Darguments=-N

as documented here: http://maven.apache.org/maven-release/maven-release-plugin/faq.html#nonrecursive

 

 

Too Many Languages!

groovy, Java, programming, Python, Scala

ARGH! I’ve been bounced between languages. It’s driving me NUTS.

This is a comparison of how to do something in each to help me map between them.

There is definitely more to each of these languages than what’s shown. But I have to start somewhere….

English:

Given a list of integers ( 1, 2, 3 ), cube each value, then filter out even numbers, then calculate the sum of the remaining numbers.

Java:

int answer = Stream.of( 1, 2, 3 )
 .map( i -> i * i * i )
 .filter( i -> i % 2 != 0 )
 .mapToInt( i -> i )
 .sum();

Groovy:

int answer = [ 1, 2, 3 ]
 .collect { it * it * it }
 .findAll { it % 2 != 0 }
 .sum()

Scala:

val answer = List( 1, 2, 3 )
 .map( i => i * i * i )
 .filter( _ % 2 != 0 )
 .sum

Python:

answer = sum([
  j for j in [
    i * i * i for i in [1, 2, 3]
  ] if j % 2 > 0
])

 

Grails—Unit Testing View Rendering (with custom TagLib namespace quirk)

Grails, programming, test

Background

GroovyPageUnitTestMixin

From the Grails guide, testing a view in a unit test involves:

  • Annotating the Specification with: @TestMixin(GroovyPageUnitTestMixin).
  • Calling render(view: '...', model: ...) or render(template: '...', model: ...) whose returned value will be a String of the HTML output.

Custom TagLibs

Normally, without doing anything else, if the view or template uses custom (i.e. not ones from the g: namespace) TagLibs, they will not be expanded. For example, if a view GSP was like this:

<div id="blah">
    <abc:decorate code="${obj}" format="a" />
</div>

the rendered content for the TagLib will be pretty much verbatim of the original GSP. The only exception is that any ${...} will be evaluated. For example:

<div id="blah">
    <abc:decorate code="Object@3234f21" format="a" />
</div>

And from the Grails guide, using mockTagLib(DecorateTagLib) (assuming the TagLib class for the abc:decorate tag is implemented in the TagLib class DecorateTagLib) will then trigger the TagLib:

@TestMixin(GroovyPageUnitTestMixin)
class MyTestSpec extends Specification {
    ...

    void testBlah() {
        given: 
        mockTagLib(DecorateTagLib)

        expect:
        ...
    }
}

will now render something like:

 
<div id="blah">
    {output of DecorateTagLib.decorate}
</div>

Note that the same effect can be accomplished using the @Mock() annotation on the class:

@TestMixin(GroovyPageUnitTestMixin)
@Mock([DecorateTagLib])
class MyTestSpec extends Specification {
    ...

    void testBlah() {
        given: 
            ...

        expect:
        ...
    }
}

And Now the Quirk…

If there were multiple TagLib classes sharing the same namespace (abc in this example) used in the GSP:
DecorateTagLib.groovy

class DecorateTagLib {
    static namespace = 'abc'

    def decorate = { attrs -> ... }
}

and FormatTagLib.groovy

class FormatTagLib {
    static namespace = 'abc'

    def format = { attrs -> ... }
}

and the GSP uses them both:

<div id="blah">
    <abc:decorate code="${obj}" format="a" />
    <abc:format val="${x}" />
</div>

and only some of them was mocked:

@TestMixin(GroovyPageUnitTestMixin)
@Mock([DecorateTagLib])
// NOTE there is no mocking of the FormatTagLib class implementing abc:format
class MyTestSpec extends Specification {
    ...

    void testBlah() {
        given: 
            ...

        expect:
        ...
    }
}

then running the test will bring up and error for the TagLibs that are not mocked. For example, the error will be like:
Tag [format] does not exist. No tag library found for namespace: abc

To fix this scenario, either:

  • Mock all the TagLibs used in the GSP that share the same namespace (e.g. abc in these examples), or
  • Redistribute the TagLibs using unique namespaces so that they do not share the same namespace. However, note that the rule stays the same for all the individual namespaces:
    • When at least one TagLib from a namespace is mocked, ALL other TagLibs used in the GSP with that namespace must also be mocked.

Some Docker Notes

docker

Some Docker notes of commands for reference

Images

Listing images

docker images

Running images

docker run <image name>
docker run <repo name>/<image name>

Building images

Create a Dockerfile file with the recipe therein.

docker build -t <image name> <parent subdir of Dockerfile>

 

Tagging images

Kinda like renaming; can be used to prepare for pushing

docker tag <image ID> <repo name>/<image name>:<version label or tag>

One version label is “latest.”

Pushing images

docker login --username=<user name> --email=<email>
docker push <image name>

Deleting images

docker rmi <repo name>
docker rmi <image ID>

The “-f” option is required if there are containers (even finished ones) referencing the image.

Containers

Kinda like processes.

Listing containers

docker ps -a

Deleting containers

docker rm <container ID>

 

 

 

 

 

Spring Boot on Scala

programming, Scala, Spring Boot, Uncategorized

Using Spring Boot on Scala

Assumption

This example uses a project generated/seeded via Activator using the “minimal-scala” template. However, any project using SBT will probably work just as well.

Add Spring Boot to dependencies

Add Spring Boot dependencies to build.sbt:

libraryDependencies += "org.springframework.boot" % "spring-boot-starter" % "1.3.2.RELEASE"
libraryDependencies += "org.springframework.boot" % "spring-boot-starter-test" % "1.3.2.RELEASE"

Create a SpringBootApplication and run it

package com.example

@SpringBootApplication
class HelloApp {

}

object Hello {
 def main(args: Array[String]): Unit = {
   SpringApplication run classOf[HelloApp]
 }
}

This class serves as a package “anchor” for component scanning. The singleton exports an equivalence of a “public static void main(String args[])” entry point.

At this point, “sbt run” will bring up the app, but it will quit soon afterward.

Time to decide if this will be a Web app or a Command line app.

Branch to Web or Command line (console)

Web app

To create a Web app, add the spring-boot-starter-web dependency. Add this to build.sbt:

libraryDependencies += "org.springframework.boot" % "spring-boot-starter-web" % "1.3.2.RELEASE"

That’s it! Now running “sbt run” will bring up the Web app listening on port 8080.

Command line app

To create a command line app (or to add a command line runner to a Web app such as above), modify the main app class:

@SpringBootApplication
class HelloApp extends CommandLineRunner {
  override def run(args: String*): Unit = {
    println("Hello from Command line runner!")
  }
}

Extending CommandLineRunner in Scala is the same as implementing CommandLineRunner in Java. So just implement the run method.

Try Autowiring something

Assuming the app class is com.example.HelloApp. The component scanning starts at com.example. So create a service class under that package (or a subpackage thereof). I will create one under com.example.services:

package com.example.services

import org.springframework.stereotype.Service

@Service
class HelloService {
  def hello(): String = {
    "Hello from HelloService!"
  }
}

Autowire into a bean using the service (HelloApp in this example):

@SpringBootApplication
class HelloApp extends CommandLineRunner {

  @Autowired
  var helloSvc: HelloService = null

  override def run(args: String*): Unit = {
    println(helloSvc.hello())
  }
}

Configuration properties

Configuration properties can go into src/main/resources/application.properties, same as in Java world:

titlemsg=Hello App

Accessing properties is also pretty much the same (e.g. shown here using @Value):

@SpringBootApplication
class HelloApp extends CommandLineRunner {
  ...

  @Value("${titlemsg}")
  var titleMsg: String = null

  override def run(args: String*): Unit = {
    println(titleMsg)
  }
}

There are several implementations of the @Value annotation available, be sure to use the one from Spring:

org.springframework.beans.factory.annotation.Value

Logging

And logging. Adding src/main/resources/logback.xml will allow tweaking of logging behavior:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/base.xml"/>
    <logger name="com.example" level="DEBUG"/>
</configuration>

I use Log4j, so here’s how to add a logger into the class to log with:

class HelloApp extends CommandLineRunner {

  val logger = Logger getLogger classOf[HelloApp]

  override def run(args: String*): Unit = {
    logger info "Hello using logger"
  }
}